00000020:00000080:2.0F:1789762539.408163:0:5786:0:(obd_config.c:826:class_setup()) finished setup of obd MGS (uuid MGS) 00000020:01000004:2.0:1789762539.408189:0:5786:0:(obd_mount.c:411:lustre_start_mgc()) Start MGC 'MGC192.168.203.148@tcp' 00000020:01000004:2.0:1789762539.408191:0:5786:0:(obd_mount.c:420:lustre_start_mgc()) mgs NIDs (null). 00000020:01000004:2.0:1789762539.408232:0:5786:0:(obd_mount.c:230:lustre_start_simple()) Starting OBD MGC192.168.203.148@tcp (typ=mgc) 00000020:00000080:2.0:1789762539.408252:0:5786:0:(genops.c:413:class_newdev()) Allocate new device MGC192.168.203.148@tcp (00000000c0af9446) 00000020:00000080:2.0:1789762539.408267:0:5786:0:(obd_config.c:720:class_attach_name()) OBD: dev 2 attached type mgc with refcount 2 00010000:00080000:2.0:1789762539.408421:0:5786:0:(ldlm_lib.c:102:import_set_conn()) imp 0000000078b811e7@MGC192.168.203.148@tcp: add connection 192.168.203.148@tcp at head 00010000:00010000:2.0:1789762539.408656:0:5786:0:(ldlm_pool.c:923:ldlm_pool_init()) Lock pool ldlm-pool-MGC192.168.203.148@tcp-0 is initialized 00000040:01000000:2.0:1789762539.408662:0:5786:0:(llog_obd.c:197:llog_setup()) obd MGC192.168.203.148@tcp ctxt 1 is initialized 10000000:01000000:2.0:1789762539.408968:0:5802:0:(mgc_request.c:631:mgc_requeue_thread()) Starting requeue thread 00000020:00000080:2.0:1789762539.408997:0:5786:0:(obd_config.c:826:class_setup()) finished setup of obd MGC192.168.203.148@tcp (uuid bbe4e1bd-8437-4448-83b9-0b3f7551a439) 00000020:00000080:2.0:1789762539.409014:0:5786:0:(genops.c:1385:class_connect()) connect: client bbe4e1bd-8437-4448-83b9-0b3f7551a439, cookie 0x416193e012b00d13 00000100:00080000:2.0:1789762539.409023:0:5786:0:(import.c:510:import_select_connection()) MGC192.168.203.148@tcp: connect to NID 0@lo last attempt 0 00000100:00080000:2.0:1789762539.409028:0:5786:0:(import.c:624:import_select_connection()) MGC192.168.203.148@tcp: import 0000000078b811e7 using connection 192.168.203.148@tcp/0@lo 00000100:00100000:2.0:1789762539.409074:0:5786:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fbfd132b x1876702022074496/t0(0) o250->MGC192.168.203.148@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762539.409127:0:5786:0:(pinger.c:373:ptlrpc_pinger_add_import()) adding pingable import bbe4e1bd-8437-4448-83b9-0b3f7551a439->MGS 00000020:01000004:2.0:1789762539.409149:0:5786:0:(tgt_mount.c:1812:server_start_targets()) starting target lustre-MDT0000 00000020:01000004:2.0:1789762539.409178:0:5786:0:(obd_mount.c:230:lustre_start_simple()) Starting OBD MDS (typ=mds) 00000020:00000080:2.0:1789762539.409190:0:5786:0:(genops.c:413:class_newdev()) Allocate new device MDS (00000000d0b93177) 00000020:00000080:2.0:1789762539.409204:0:5786:0:(obd_config.c:720:class_attach_name()) OBD: dev 3 attached type mds with refcount 2 00000100:00100000:2.0:1789762539.415297:0:5786:0:(service.c:155:ptlrpc_grow_req_bufs()) mdt: allocate 64 new 262144-byte reqbufs (0/64 left), rc = 0 00000100:00100000:2.0:1789762539.415506:0:5786:0:(service.c:3462:ptlrpc_start_thread()) mdt[0] started 0 min 3 max 116 00000100:00100000:2.0:1789762539.415513:0:5786:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt00_000' 00000100:00100000:2.0:1789762539.416300:0:5786:0:(service.c:3462:ptlrpc_start_thread()) mdt[0] started 2 min 3 max 116 00000100:00100000:2.0:1789762539.416308:0:5786:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt00_002' 00000100:00100000:2.0:1789762539.417267:0:5786:0:(service.c:3462:ptlrpc_start_thread()) mdt_readpage[0] started 1 min 2 max 80 00000100:00100000:2.0:1789762539.417275:0:5786:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt_rdpg00_001' 00000100:00100000:2.0:1789762539.438404:0:5786:0:(service.c:3462:ptlrpc_start_thread()) mdt_out[0] started 1 min 2 max 116 00000100:00100000:2.0:1789762539.438414:0:5786:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt_out00_001' 00000100:00100000:2.0:1789762539.439177:0:5786:0:(service.c:3462:ptlrpc_start_thread()) mdt_seqs[-1] started 1 min 2 max 256 00000100:00100000:2.0:1789762539.439184:0:5786:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt_seqs_0001' 00000100:00100000:2.0:1789762539.439864:0:5786:0:(service.c:3462:ptlrpc_start_thread()) mdt_seqm[-1] started 1 min 2 max 256 00000100:00100000:2.0:1789762539.439877:0:5786:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt_seqm_0001' 00000100:00100000:2.0:1789762539.440539:0:5786:0:(service.c:3462:ptlrpc_start_thread()) mdt_fld[-1] started 1 min 2 max 256 00000100:00100000:2.0:1789762539.440546:0:5786:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt_fld_0001' 00010000:00010000:2.0:1789762539.447508:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762539.454239:0:5786:0:(service.c:3462:ptlrpc_start_thread()) mdt_io[0] started 1 min 3 max 116 00000100:00100000:2.0:1789762539.454248:0:5786:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt_io00_001' 00000020:00000080:2.0:1789762539.455761:0:5786:0:(obd_config.c:826:class_setup()) finished setup of obd MDS (uuid MDS_uuid) 00000020:01000004:2.0:1789762539.455782:0:5786:0:(tgt_mount.c:280:server_mgc_set_fs()) Set mgc disk for lustre-mdt1/mdt1 10000000:01000000:2.0:1789762539.455788:0:5786:0:(mgc_request_server.c:89:mgc_fs_setup()) MGC192.168.203.148@tcp: cl_mgc_mutex lustre-MDT0000-osd for locked: rc = 0 00000040:01000000:2.0:1789762539.455934:0:5786:0:(llog_obd.c:197:llog_setup()) obd MGC192.168.203.148@tcp ctxt 0 is initialized 00000020:01000004:2.0:1789762539.455964:0:5786:0:(tgt_mount.c:1418:server_register_target()) Registration lustre-MDT0000, fs=lustre, 192.168.203.148@tcp, index=0000, flags=0x20065 00000020:01000004:2.0:1789762539.455969:0:5786:0:(tgt_mount.c:1175:server_mti_print()) mti - server_register_target 00000020:01000004:2.0:1789762539.455971:0:5786:0:(tgt_mount.c:1176:server_mti_print()) server: lustre-MDT0000 00000020:01000004:2.0:1789762539.455972:0:5786:0:(tgt_mount.c:1177:server_mti_print()) fs: lustre 00000020:01000004:2.0:1789762539.455974:0:5786:0:(tgt_mount.c:1178:server_mti_print()) uuid: 00000020:01000004:2.0:1789762539.455976:0:5786:0:(tgt_mount.c:1179:server_mti_print()) ver: 0 00000020:01000004:2.0:1789762539.455977:0:5786:0:(tgt_mount.c:1180:server_mti_print()) flags: 00000020:01000004:2.0:1789762539.455979:0:5786:0:(tgt_mount.c:1182:server_mti_print()) LDD_F_SV_TYPE_MDT 00000020:01000004:2.0:1789762539.455980:0:5786:0:(tgt_mount.c:1186:server_mti_print()) LDD_F_SV_TYPE_MGS 00000020:01000004:2.0:1789762539.455981:0:5786:0:(tgt_mount.c:1192:server_mti_print()) LDD_F_VIRIGIN 00000020:01000004:2.0:1789762539.455983:0:5786:0:(tgt_mount.c:1194:server_mti_print()) LDD_F_UPDATE 00000020:01000004:2.0:1789762539.455984:0:5786:0:(tgt_mount.c:1214:server_mti_print()) LDD_F_LARGE_NID 00000020:01000004:2.0:1789762539.455985:0:5786:0:(tgt_mount.c:1216:server_mti_print()) LDD_F_OPC_REG 10000000:01000000:2.0:1789762539.455988:0:5786:0:(mgc_request_server.c:451:mgc_set_info_async_server()) register_target lustre-MDT0000 0x10020065 00000020:01000004:2.0:1789762539.455990:0:5786:0:(tgt_mount.c:1175:server_mti_print()) mti - mgc_target_register: req 00000020:01000004:2.0:1789762539.455992:0:5786:0:(tgt_mount.c:1176:server_mti_print()) server: lustre-MDT0000 00000020:01000004:2.0:1789762539.455993:0:5786:0:(tgt_mount.c:1177:server_mti_print()) fs: lustre 00000020:01000004:2.0:1789762539.455994:0:5786:0:(tgt_mount.c:1178:server_mti_print()) uuid: 00000020:01000004:2.0:1789762539.455995:0:5786:0:(tgt_mount.c:1179:server_mti_print()) ver: 0 00000020:01000004:2.0:1789762539.455997:0:5786:0:(tgt_mount.c:1180:server_mti_print()) flags: 00000020:01000004:2.0:1789762539.455998:0:5786:0:(tgt_mount.c:1182:server_mti_print()) LDD_F_SV_TYPE_MDT 00000020:01000004:2.0:1789762539.455999:0:5786:0:(tgt_mount.c:1186:server_mti_print()) LDD_F_SV_TYPE_MGS 00000020:01000004:2.0:1789762539.456000:0:5786:0:(tgt_mount.c:1192:server_mti_print()) LDD_F_VIRIGIN 00000020:01000004:2.0:1789762539.456001:0:5786:0:(tgt_mount.c:1194:server_mti_print()) LDD_F_UPDATE 00000020:01000004:2.0:1789762539.456002:0:5786:0:(tgt_mount.c:1214:server_mti_print()) LDD_F_LARGE_NID 00000020:01000004:2.0:1789762539.456003:0:5786:0:(tgt_mount.c:1216:server_mti_print()) LDD_F_OPC_REG 10000000:01000000:2.0:1789762539.456026:0:5786:0:(mgc_request_server.c:281:mgc_target_register()) register lustre-MDT0000 00000100:00100000:2.0:1789762539.456041:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022074624:0@lo:253: 00000100:00100000:2.0:1789762539.456145:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.456196:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 00000000ba7856a6 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.533544:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022074624:0@lo:253: 10000000:01000000:2.0:1789762539.533560:0:5786:0:(mgc_request_server.c:302:mgc_target_register()) register lustre-MDT0000 got index = 0 00000020:01000004:2.0:1789762539.533562:0:5786:0:(tgt_mount.c:1175:server_mti_print()) mti - mgc_target_register: rep 00000020:01000004:2.0:1789762539.533564:0:5786:0:(tgt_mount.c:1176:server_mti_print()) server: lustre-MDT0000 00000020:01000004:2.0:1789762539.533565:0:5786:0:(tgt_mount.c:1177:server_mti_print()) fs: lustre 00000020:01000004:2.0:1789762539.533566:0:5786:0:(tgt_mount.c:1178:server_mti_print()) uuid: lustre-MDT0000_UUID 00000020:01000004:2.0:1789762539.533567:0:5786:0:(tgt_mount.c:1179:server_mti_print()) ver: 0 00000020:01000004:2.0:1789762539.533568:0:5786:0:(tgt_mount.c:1180:server_mti_print()) flags: 00000020:01000004:2.0:1789762539.533570:0:5786:0:(tgt_mount.c:1182:server_mti_print()) LDD_F_SV_TYPE_MDT 00000020:01000004:2.0:1789762539.533571:0:5786:0:(tgt_mount.c:1186:server_mti_print()) LDD_F_SV_TYPE_MGS 00000020:01000004:2.0:1789762539.533572:0:5786:0:(tgt_mount.c:1196:server_mti_print()) LDD_F_REWRITE_LDD 00000020:01000004:2.0:1789762539.533573:0:5786:0:(tgt_mount.c:1214:server_mti_print()) LDD_F_LARGE_NID 00000020:01000004:2.0:1789762539.533574:0:5786:0:(tgt_mount.c:1216:server_mti_print()) LDD_F_OPC_REG 00000020:01000004:2.0:1789762539.533590:0:5786:0:(tgt_mount.c:101:server_register_mount()) register mount 000000001e15a5ed from lustre-MDT0000 10000000:01000000:2.0:1789762539.533608:0:5786:0:(mgc_request.c:1944:mgc_process_config()) parse_log lustre-MDT0000 from 0 10000000:01000000:2.0:1789762539.533610:0:5786:0:(mgc_request.c:329:config_log_add()) add config log lustre-MDT0000-0000000000000000 10000000:01000000:2.0:1789762539.533613:0:5786:0:(mgc_request.c:194:do_config_log_add()) do adding config log lustre-sptlrpc-ffff981e851899b8 10000000:01000000:2.0:1789762539.533616:0:5786:0:(mgc_request.c:68:mgc_name2resid()) log lustre-sptlrpc to resid 0x65727473756c/0x0 (lustre) 10000000:01000000:2.0:1789762539.533622:0:5786:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff981e851899b8 from 1 10000000:01000000:2.0:1789762539.533624:0:5786:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1789762539.533650:0:5786:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.533658:0:5786:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.533665:0:5786:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.533676:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022074752:0@lo:101: 00000100:00100000:2.0:1789762539.533719:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.533763:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 000000004435fbd2 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.533947:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022074752:0@lo:101: 00010000:00010000:2.0:1789762539.533961:0:5786:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.533968:0:5786:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.534000:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022074880:0@lo:501: 00000100:00100000:2.0:1789762539.534020:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.534036:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 000000004435fbd2 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.534291:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022074880:0@lo:501: 10000000:01000000:2.0:1789762539.534327:0:5786:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.148@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:2.0:1789762539.534337:0:5786:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.534343:0:5786:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762539.534348:0:5786:0:(mgc_request.c:194:do_config_log_add()) do adding config log params-ffff981e903ad000 10000000:01000000:2.0:1789762539.534350:0:5786:0:(mgc_request.c:68:mgc_name2resid()) log params to resid 0x736d61726170/0x3 (params) 10000000:01000000:2.0:1789762539.534353:0:5786:0:(mgc_request.c:194:do_config_log_add()) do adding config log lustre-barrier-ffff981e903ad000 10000000:01000000:2.0:1789762539.534355:0:5786:0:(mgc_request.c:68:mgc_name2resid()) log lustre-barrier to resid 0x65727473756c/0x5 (lustre) 10000000:01000000:2.0:1789762539.534357:0:5786:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-barrier-ffff981e903ad000 from 1 10000000:01000000:2.0:1789762539.534358:0:5786:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-barrier (res 0x65727473756c) 00010000:00010000:2.0:1789762539.534369:0:5786:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.148@tcp lock: 00000000ec38eb78/0x416193e012b00d3d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.534382:0:5786:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.148@tcp lock: 00000000ec38eb78/0x416193e012b00d3d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.534388:0:5786:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.148@tcp lock: 00000000ec38eb78/0x416193e012b00d3d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.534396:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075008:0@lo:101: 00000100:00100000:2.0:1789762539.534416:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.534433:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 000000004435fbd2 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.534591:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075008:0@lo:101: 00010000:00010000:2.0:1789762539.534601:0:5786:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.148@tcp lock: 00000000ec38eb78/0x416193e012b00d3d lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00d44 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.534607:0:5786:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.148@tcp lock: 00000000ec38eb78/0x416193e012b00d3d lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x416193e012b00d44 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762539.534615:0:5786:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.148@tcp: configuration from log 'lustre-barrier' succeeded (0). 00010000:00010000:2.0:1789762539.534623:0:5786:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.148@tcp lock: 00000000ec38eb78/0x416193e012b00d3d lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00d44 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.534627:0:5786:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.148@tcp lock: 00000000ec38eb78/0x416193e012b00d3d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00d44 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762539.534631:0:5786:0:(mgc_request.c:194:do_config_log_add()) do adding config log lustre-MDT0000-0000000000000000 10000000:01000000:2.0:1789762539.534634:0:5786:0:(mgc_request.c:68:mgc_name2resid()) log lustre-MDT0000 to resid 0x65727473756c/0x0 (lustre) 10000000:01000000:2.0:1789762539.534637:0:5786:0:(mgc_request.c:194:do_config_log_add()) do adding config log lustre-mdtir-ffff981e903ad000 10000000:01000000:2.0:1789762539.534639:0:5786:0:(mgc_request.c:68:mgc_name2resid()) log lustre-mdtir to resid 0x65727473756c/0x2 (lustre) 10000000:01000000:2.0:1789762539.534640:0:5786:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-MDT0000-0000000000000000 from 1 10000000:01000000:2.0:1789762539.534647:0:5786:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-MDT0000 (res 0x65727473756c) 00010000:00010000:2.0:1789762539.534658:0:5786:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.148@tcp lock: 00000000e50e4b76/0x416193e012b00d4b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.534662:0:5786:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.148@tcp lock: 00000000e50e4b76/0x416193e012b00d4b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.534667:0:5786:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.148@tcp lock: 00000000e50e4b76/0x416193e012b00d4b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.534674:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075136:0@lo:101: 00000100:00100000:2.0:1789762539.534692:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.534704:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 000000004435fbd2 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.534867:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075136:0@lo:101: 00010000:00010000:2.0:1789762539.534879:0:5786:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.148@tcp lock: 00000000e50e4b76/0x416193e012b00d4b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00d52 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.534888:0:5786:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.148@tcp lock: 00000000e50e4b76/0x416193e012b00d4b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x416193e012b00d52 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.534921:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075264:0@lo:501: 00000100:00100000:2.0:1789762539.534944:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.534956:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 000000004435fbd2 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.535176:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075264:0@lo:501: 00000100:00100000:2.0:1789762539.535199:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075392:0@lo:503: 00000100:00100000:2.0:1789762539.535228:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.535249:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 000000004435fbd2 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.535454:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075392:0@lo:503: 00080000:10000000:2.0:1789762539.635411:0:5827:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000006:0x1020000:0x0] (8/8/4) registered 00000020:01000000:2.0:1789762539.655918:0:5786:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-MDT0000 gen 1-15 (rc=0) 10000000:01000000:2.0:1789762539.655934:0:5786:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.148@tcp: configuration from log 'lustre-MDT0000' succeeded (0). 00010000:00010000:2.0:1789762539.655948:0:5786:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.148@tcp lock: 00000000e50e4b76/0x416193e012b00d4b lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00d52 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.655957:0:5786:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.148@tcp lock: 00000000e50e4b76/0x416193e012b00d4b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00d52 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762539.655964:0:5786:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-mdtir-ffff981e903ad000 from 1 10000000:01000000:2.0:1789762539.655966:0:5786:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-mdtir (res 0x65727473756c) 00010000:00010000:2.0:1789762539.655986:0:5786:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.148@tcp lock: 0000000025ce36b3/0x416193e012b00d9f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.655991:0:5786:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.148@tcp lock: 0000000025ce36b3/0x416193e012b00d9f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.655997:0:5786:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.148@tcp lock: 0000000025ce36b3/0x416193e012b00d9f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.656006:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075776:0@lo:101: 00000100:00100000:2.0:1789762539.656030:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.656043:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 0000000075283600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.656225:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075776:0@lo:101: 00010000:00010000:2.0:1789762539.656236:0:5786:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.148@tcp lock: 0000000025ce36b3/0x416193e012b00d9f lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00da6 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.656242:0:5786:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.148@tcp lock: 0000000025ce36b3/0x416193e012b00d9f lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x416193e012b00da6 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.656726:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075904:0@lo:256: 00000100:00100000:2.0:1789762539.656761:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.656785:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 0000000075283600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.656960:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022075904:0@lo:256: 10000000:01000000:2.0:1789762539.657475:0:5786:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.148@tcp: configuration from log 'lustre-mdtir' succeeded (0). 00010000:00010000:2.0:1789762539.657486:0:5786:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.148@tcp lock: 0000000025ce36b3/0x416193e012b00d9f lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00da6 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.657491:0:5786:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.148@tcp lock: 0000000025ce36b3/0x416193e012b00d9f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00da6 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762539.657495:0:5786:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff981e903ad000 from 1 10000000:01000000:2.0:1789762539.657497:0:5786:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1789762539.657509:0:5786:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.148@tcp lock: 00000000d82fed4c/0x416193e012b00dad lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.657513:0:5786:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.148@tcp lock: 00000000d82fed4c/0x416193e012b00dad lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.657517:0:5786:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.148@tcp lock: 00000000d82fed4c/0x416193e012b00dad lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.657525:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076032:0@lo:101: 00000100:00100000:2.0:1789762539.657543:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.657559:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 0000000075283600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.657724:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076032:0@lo:101: 00010000:00010000:2.0:1789762539.657734:0:5786:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.148@tcp lock: 00000000d82fed4c/0x416193e012b00dad lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00db4 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.657744:0:5786:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.148@tcp lock: 00000000d82fed4c/0x416193e012b00dad lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x416193e012b00db4 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.657771:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076160:0@lo:501: 00000100:00100000:2.0:1789762539.657789:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.657801:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 0000000075283600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.658190:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076160:0@lo:501: 00000100:00100000:2.0:1789762539.658212:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076288:0@lo:503: 00000100:00100000:2.0:1789762539.658242:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.658260:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 0000000075283600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.658407:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076288:0@lo:503: 00000020:01000000:2.0:1789762539.659017:0:5786:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 1-0 (rc=0) 10000000:01000000:2.0:1789762539.659032:0:5786:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.148@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:2.0:1789762539.659040:0:5786:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.148@tcp lock: 00000000d82fed4c/0x416193e012b00dad lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00db4 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.659046:0:5786:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.148@tcp lock: 00000000d82fed4c/0x416193e012b00dad lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00db4 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762539.659070:0:5786:0:(mgc_request.c:1944:mgc_process_config()) parse_log lustre-client from 0 10000000:01000000:2.0:1789762539.659072:0:5786:0:(mgc_request.c:329:config_log_add()) add config log lustre-client-ffff981e903ad000 10000000:01000000:2.0:1789762539.659075:0:5786:0:(mgc_request.c:194:do_config_log_add()) do adding config log lustre-client-ffff981e903ad000 10000000:01000000:2.0:1789762539.659077:0:5786:0:(mgc_request.c:68:mgc_name2resid()) log lustre-client to resid 0x65727473756c/0x0 (lustre) 10000000:01000000:2.0:1789762539.659080:0:5786:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff981e903ad000 from 1 10000000:01000000:2.0:1789762539.659081:0:5786:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1789762539.659116:0:5786:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.148@tcp lock: 000000000103a2c3/0x416193e012b00dbb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.659126:0:5786:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.148@tcp lock: 000000000103a2c3/0x416193e012b00dbb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.659131:0:5786:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.148@tcp lock: 000000000103a2c3/0x416193e012b00dbb lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.659139:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076416:0@lo:101: 00000100:00100000:2.0:1789762539.659157:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.659181:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 0000000075283600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.659323:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076416:0@lo:101: 00010000:00010000:2.0:1789762539.659331:0:5786:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.148@tcp lock: 000000000103a2c3/0x416193e012b00dbb lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00dc2 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.659336:0:5786:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.148@tcp lock: 000000000103a2c3/0x416193e012b00dbb lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x416193e012b00dc2 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.659365:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076544:0@lo:501: 00000100:00100000:2.0:1789762539.659391:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.659408:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 0000000075283600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.659594:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076544:0@lo:501: 00000100:00100000:2.0:1789762539.659614:0:5786:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076672:0@lo:503: 00000100:00100000:2.0:1789762539.659639:0:5786:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.659652:0:5786:0:(client.c:2715:ptlrpc_set_wait()) set 0000000075283600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.659944:0:5786:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mount.lustre:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5786:1876702022076672:0@lo:503: 20000000:01000000:2.0:1789762539.672556:0:5820:0:(mgs_nids.c:712:mgs_ir_notify()) mgs_lustre_notify woken up, phase is 1 10000000:01000000:2.0:1789762539.672566:0:5820:0:(mgc_request.c:68:mgc_name2resid()) log lustre to resid 0x65727473756c/0x2 (lustre) 00010000:00010000:2.0:1789762539.672579:0:5820:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: 000000003dd12418/0x416193e012b00dd7 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 5820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.672592:0:5820:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: 000000007d7e8710/0x416193e012b00da6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b00d9f expref: 12 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.672598:0:5820:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: 000000003dd12418/0x416193e012b00dd7 lrc: 4/0,1 mode: --/EX res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 5820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.672622:0:5820:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: 000000007d7e8710/0x416193e012b00da6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x416193e012b00d9f expref: 12 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762539.672633:0:5820:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: 000000007d7e8710/0x416193e012b00da6 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x416193e012b00d9f expref: 12 pid: 5798 timeout: 212 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.672643:0:5820:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mgs_lustre_noti:MGS:5820:1876702022077312:0@lo:104: 00000100:00100000:2.0:1789762539.672682:0:5820:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762539.672720:0:5820:0:(client.c:2715:ptlrpc_set_wait()) set 0000000075283600 going to sleep for 11 seconds 00000100:00100000:2.0:1789762539.672851:0:5820:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000002e7b86de pname:cluuid:pid:xid:nid:opc:job mgs_lustre_noti:MGS:5820:1876702022077312:0@lo:104: 00010000:00010000:2.0:1789762539.672875:0:5820:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: 000000003dd12418/0x416193e012b00dd7 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 5820 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762539.673043:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@00000000cbfe639f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022077440:0@lo:103: 00000100:00100000:2.0:1789762539.673068:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00080000:10000000:2.0:1789762539.679547:0:5786:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000003:0xd:0x0] (8/8/4) registered 00080000:10000000:2.0:1789762539.679666:0:5786:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000005:0x20001:0x0] (8/8/1) registered 00080000:10000000:2.0:1789762539.679793:0:5786:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000003:0xe:0x0] (8/8/4) registered 00080000:10000000:2.0:1789762539.679939:0:5786:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000005:0x20002:0x0] (8/8/1) registered 00080000:10000000:2.0:1789762539.680075:0:5786:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000003:0xf:0x0] (8/8/4) registered 00080000:10000000:2.0:1789762539.680241:0:5786:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000005:0x20003:0x0] (8/8/1) registered 00000040:01000000:2.0:1789762539.707258:0:5786:0:(llog_obd.c:197:llog_setup()) obd lustre-MDT0000-osd ctxt 16 is initialized 00000040:01000000:2.0:1789762539.711905:0:5786:0:(llog_obd.c:197:llog_setup()) obd lustre-MDD0000 ctxt 12 is initialized 00000040:00080000:2.0:1789762539.712684:0:5786:0:(llog_osd.c:211:llog_osd_read_header()) not reading header from 0-byte log 00000004:00000080:2.0:1789762539.713274:0:5786:0:(mdd_device.c:555:mdd_changelog_llog_init()) changelog starting index=0 00000040:01000000:2.0:1789762539.713286:0:5786:0:(llog_obd.c:197:llog_setup()) obd lustre-MDD0000 ctxt 14 is initialized 00000040:00080000:2.0:1789762539.713518:0:5786:0:(llog_osd.c:211:llog_osd_read_header()) not reading header from 0-byte log 00000040:00080000:2.0:1789762539.714777:0:5786:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x5:0x0] index 64768 count 1 00000040:00080000:2.0:1789762539.715519:0:5786:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x6:0x0] index 64768 count 1 00000040:01000000:2.0:1789762539.715567:0:5786:0:(llog_obd.c:197:llog_setup()) obd lustre-MDD0000 ctxt 15 is initialized 00000040:00080000:2.0:1789762539.715806:0:5786:0:(llog_osd.c:211:llog_osd_read_header()) not reading header from 0-byte log 00080000:10000000:2.0:1789762539.716230:0:5786:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000003:0x11:0x0] (8/8/4) registered 00100000:10000000:2.0:1789762539.718416:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_00: rc = 0 00100000:10000000:2.0:1789762539.718705:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_01: rc = 0 00100000:10000000:2.0:1789762539.718937:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_02: rc = 0 00100000:10000000:2.0:1789762539.719199:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_03: rc = 0 00100000:10000000:2.0:1789762539.719454:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_04: rc = 0 00100000:10000000:2.0:1789762539.719718:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_05: rc = 0 00100000:10000000:2.0:1789762539.720007:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_06: rc = 0 00100000:10000000:2.0:1789762539.720497:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_07: rc = 0 00100000:10000000:2.0:1789762539.720857:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_08: rc = 0 00100000:10000000:2.0:1789762539.721238:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_09: rc = 0 00100000:10000000:2.0:1789762539.721632:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_10: rc = 0 00100000:10000000:2.0:1789762539.721968:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_11: rc = 0 00100000:10000000:2.0:1789762539.722322:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_12: rc = 0 00100000:10000000:2.0:1789762539.722661:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_13: rc = 0 00100000:10000000:2.0:1789762539.723025:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_14: rc = 0 00100000:10000000:2.0:1789762539.723518:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_namespace_15: rc = 0 00100000:10000000:2.0:1789762539.724474:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_00: rc = 0 00100000:10000000:2.0:1789762539.724834:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_01: rc = 0 00100000:10000000:2.0:1789762539.725211:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_02: rc = 0 00100000:10000000:2.0:1789762539.725624:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_03: rc = 0 00100000:10000000:2.0:1789762539.726030:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_04: rc = 0 00100000:10000000:2.0:1789762539.726429:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_05: rc = 0 00100000:10000000:2.0:1789762539.726741:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_06: rc = 0 00100000:10000000:2.0:1789762539.727217:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_07: rc = 0 00100000:10000000:2.0:1789762539.727634:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_08: rc = 0 00100000:10000000:2.0:1789762539.728051:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_09: rc = 0 00100000:10000000:2.0:1789762539.728459:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_10: rc = 0 00100000:10000000:2.0:1789762539.728862:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_11: rc = 0 00100000:10000000:2.0:1789762539.729260:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_12: rc = 0 00100000:10000000:2.0:1789762539.729584:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_13: rc = 0 00100000:10000000:2.0:1789762539.729839:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_14: rc = 0 00100000:10000000:2.0:1789762539.730152:0:5786:0:(lfsck_lib.c:2686:lfsck_load_one_trace_file()) lustre-MDT0000-osd: unlink lfsck sub trace file lfsck_layout_15: rc = 0 00000001:00000004:2.0:1789762539.730532:0:5786:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-MDT0000_UUID: mount_count is 1, last_transno is 4294967296 00000001:00080000:2.0:1789762539.751989:0:5786:0:(tgt_lastrcvd.c:222:tgt_reply_header_read()) lustre-MDT0000: read reply_data header. magic=0xbdabda01 header_size=32 reply_size=32 00000004:00000080:2.0:1789762539.752018:0:5786:0:(mdd_device.c:2304:mdd_iocontrol()) lustre-MDD0000: cmd=c00866e6 len=0 karg=ffffb40c01cc7988 00100000:10000000:2.0:1789762539.752026:0:5786:0:(lfsck_lib.c:3085:lfsck_start()) lustre-MDT0000-osd: start master thread 00000020:01000004:2.0:1789762539.752304:0:5786:0:(tgt_mount.c:298:server_mgc_clear_fs()) Unassign mgc disk 10000000:01000000:2.0:1789762539.752334:0:5786:0:(mgc_request_server.c:166:mgc_fs_clear()) MGC192.168.203.148@tcp: cl_mgc_mutex unlock 00000020:01000004:2.0:1789762539.752352:0:5786:0:(tgt_mount.c:2461:server_fill_super_common()) Server sb, dev=45 00000080:00000004:2.0:1789762539.752369:0:5786:0:(super25.c:182:lustre_fill_super()) lustre-mdt1/mdt1: Mount complete 00010000:00010000:2.0:1789762540.471220:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789762541.002791:0:6081:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 00010000:00010000:2.0:1789762542.519222:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1789762543.223928:0:6282:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=400866bb len=0 uarg=00007fff349a0ce0 20000000:00000080:2.0:1789762543.223940:0:6282:0:(mgs_handler.c:1167:mgs_iocontrol()) MGS: cmd=400866bb len=576 karg=ffff981eb499f000 uarg=0000000000000000 20000000:01000000:2.0:1789762543.223963:0:6282:0:(mgs_llog.c:6192:mgs_set_param2()) set_param2 fs='' device='general' param='debug_raw_pointers=Y' 20000000:01000000:2.0:1789762543.223967:0:6282:0:(mgs_llog.c:4647:mgs_write_log_param2()) next param 'debug_raw_pointers=Y' 20000000:01000000:2.0:1789762543.223971:0:6282:0:(mgs_llog.c:1379:mgs_modify_param()) modify params/general/debug_raw_pointers=Y del=0 00000040:00080000:2.0:1789762543.224252:0:6282:0:(llog_osd.c:211:llog_osd_read_header()) not reading header from 0-byte log 20000000:02000000:2.0:1789762543.224260:0:6282:0:(mgs_llog.c:4120:mgs_wlp_lcfg()) Modifying parameter general.debug_raw_pointers=Y in log params 00000040:00080000:2.0:1789762543.226454:0:6282:0:(llog_osd.c:211:llog_osd_read_header()) not reading header from 0-byte log 00000040:00080000:2.0:1789762543.226627:0:6282:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x2:0x0].1, 224 off8416 00000040:00080000:2.0:1789762543.226666:0:6282:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x2:0x0].2, 112 off8528 00000040:00080000:2.0:1789762543.226694:0:6282:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x2:0x0].3, 224 off8752 10000000:01000000:2.0:1789762543.226703:0:6282:0:(mgc_request.c:68:mgc_name2resid()) log params to resid 0x736d61726170/0x3 (params) 00010000:00010000:2.0:1789762543.226717:0:6282:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: 000000003dd12418/0x416193e012b00dde lrc: 3/0,1 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6282 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762543.226731:0:6282:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: 00000000492c89fe/0x416193e012b00db4 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b00dad expref: 10 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762543.226737:0:6282:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: 000000003dd12418/0x416193e012b00dde lrc: 4/0,1 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6282 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762543.226765:0:6282:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: 00000000492c89fe/0x416193e012b00db4 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x416193e012b00dad expref: 10 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762543.226774:0:6282:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: 00000000492c89fe/0x416193e012b00db4 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x416193e012b00dad expref: 10 pid: 5798 timeout: 216 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762543.226784:0:6282:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000005feb9895 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:6282:1876702022077568:0@lo:104: 00000100:00100000:2.0:1789762543.226831:0:6282:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762543.226876:0:6282:0:(client.c:2715:ptlrpc_set_wait()) set 00000000c2e48703 going to sleep for 11 seconds 00000100:00100000:2.0:1789762543.227030:0:6282:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000005feb9895 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:6282:1876702022077568:0@lo:104: 00010000:00010000:2.0:1789762543.227052:0:6282:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: 000000003dd12418/0x416193e012b00dde lrc: 3/0,1 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6282 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762543.227221:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@0000000028199d0d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022077696:0@lo:103: 00000100:00100000:2.0:1789762543.227254:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789762543.543204:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762544.759255:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762544.759285:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@000000007ef7e6ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022077824:0@lo:400: 00000100:00100000:2.0:1789762544.759351:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762544.759425:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022077952 00000100:00100000:2.0:1789762544.759430:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022077952 00000100:00100000:2.0:1789762544.759464:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1 00000100:00100000:2.0:1789762544.759470:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@00000000633c308a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3270:x1876702022077952:12345-0@lo:38: 00010000:00080000:2.0:1789762544.759493:0:5806:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-MDT0000: connection from lustre-MDT0000-lwp-MDT0000_UUID@0@lo t0 exp 0000000000000000 cur 68 last 0 00000020:00000080:2.0:1789762544.759526:0:5806:0:(genops.c:1385:class_connect()) connect: client lustre-MDT0000-lwp-MDT0000_UUID, cookie 0x416193e012b00de5 00000004:00100000:2.0:1789762544.759577:0:5806:0:(mdt_handler.c:7297:mdt_connect_internal()) lustre-MDT0000: cli does not support OBD_CONNECT_CKSUM, CRC32 will be used 00000100:00080000:2.0:1789762544.759711:0:5806:0:(import.c:65:import_set_state_nolock()) 00000000685d5104 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789762544.759755:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@00000000633c308a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3270:x1876702022077952:12345-0@lo:38: Request processed in 285us (365us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762544.759762:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1 00000100:00100000:2.0:1789762544.759804:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@000000007ef7e6ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022077824:0@lo:400: 00000040:00080000:2.0:1789762545.112478:0:6661:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x8:0x0] index 5 count 5 00000100:00100000:2.0:1789762545.112905:0:5789:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2 00000100:00100000:2.0:1789762545.112915:0:5789:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@0000000037e49224 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+4:5798:x1876702022078208:12345-0@lo:104: 00010000:00010000:2.0:1789762545.112926:0:5789:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.148@tcp lock: 000000000103a2c3/0x416193e012b00dbb lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00dc2 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.112983:0:5789:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1789762545.112996:0:5789:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@0000000037e49224 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+4:5798:x1876702022078208:12345-0@lo:104: Request processed in 83us (200us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.113010:0:5789:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2 00000100:00100000:2.0:1789762545.113034:0:5789:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022078464 00000100:00100000:2.0:1789762545.113040:0:5789:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022078464 00000100:00100000:2.0:1789762545.113064:0:5789:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 3 00000100:00100000:2.0:1789762545.113072:0:5789:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@0000000058c4d071 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+4:5798:x1876702022078336:12345-0@lo:104: 00010000:00010000:2.0:1789762545.113081:0:5789:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.148@tcp lock: 00000000e50e4b76/0x416193e012b00d4b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00d52 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.113171:0:5789:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1789762545.113178:0:5789:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@0000000058c4d071 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+4:5798:x1876702022078336:12345-0@lo:104: Request processed in 107us (320us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.113193:0:5789:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 3 00000100:00100000:2.0:1789762545.113214:0:5789:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 4 00000100:00100000:2.0:1789762545.113227:0:5789:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@00000000da9dce5c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+3:5798:x1876702022078464:12345-0@lo:104: 00010000:00010000:2.0:1789762545.113240:0:5789:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.113303:0:5789:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1789762545.113315:0:5789:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@00000000da9dce5c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+3:5798:x1876702022078464:12345-0@lo:104: Request processed in 91us (384us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.113326:0:5789:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 4 00010000:00010000:2.0:1789762545.113352:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762545.113574:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762545.113582:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.113600:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock 00000000d165cc67 already unused, calling callback (0000000009722d31) 10000000:00010000:2.0:1789762545.113611:0:5793:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.113620:0:5793:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789762545.113626:0:5793:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762545.113633:0:5793:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789762545.113660:0:5793:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.113668:0:5793:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762545.113687:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.113700:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.148@tcp lock: 00000000d165cc67/0x416193e012b00d2f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00d36 expref: -99 pid: 5786 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.113730:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762545.113733:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762545.113736:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762545.113744:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762545.113747:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762545.125155:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@000000006b3952f4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+7:6644:x1876702022078080:12345-0@lo:253: Request processed in 16377us (16508us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.125165:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22 00000100:00100000:2.0:1789762545.125641:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022079104 00000100:00100000:2.0:1789762545.125646:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022079104 00000100:00100000:2.0:1789762545.125666:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24 00000100:00100000:2.0:1789762545.125676:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@00000000acad00b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+8:6644:x1876702022079104:12345-0@lo:501: 00000100:00100000:2.0:1789762545.125775:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@00000000acad00b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+8:6644:x1876702022079104:12345-0@lo:501: Request processed in 100us (167us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789762545.125782:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24 00010000:00010000:2.0:1789762545.125894:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762545.125899:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.148@tcp lock: 00000000df50b444/0x416193e012b00e08 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x400000000 nid: local remote: 0x416193e012b00e0f expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.125911:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock 00000000df50b444 already unused, calling callback (0000000009722d31) 10000000:00010000:2.0:1789762545.125914:0:5794:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.148@tcp lock: 00000000df50b444/0x416193e012b00e08 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x400000000 nid: local remote: 0x416193e012b00e0f expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.125924:0:5794:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.148@tcp lock: 00000000df50b444/0x416193e012b00e08 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x8400000000 nid: local remote: 0x416193e012b00e0f expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789762545.125931:0:5794:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.148@tcp lock: 00000000df50b444/0x416193e012b00e08 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x9400000000 nid: local remote: 0x416193e012b00e0f expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762545.125934:0:5794:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789762545.125948:0:5794:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.148@tcp lock: 00000000df50b444/0x416193e012b00e08 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x4809400000020 nid: local remote: 0x416193e012b00e0f expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.125960:0:5794:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762545.125979:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.148@tcp lock: 00000000df50b444/0x416193e012b00e08 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x4809400000020 nid: local remote: 0x416193e012b00e0f expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.125988:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.148@tcp lock: 00000000df50b444/0x416193e012b00e08 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x4809400000020 nid: local remote: 0x416193e012b00e0f expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.126005:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762545.126014:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762545.126020:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762545.126121:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022079360 00000100:00100000:2.0:1789762545.126128:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022079360 00000100:00100000:2.0:1789762545.126162:0:6664:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 0 00000100:00100000:2.0:1789762545.126172:0:6664:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@000000009b01ba36 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:bbe4e1bd-8437-4448-83b9-0b3f7551a439+10:3274:x1876702022079360:12345-0@lo:103: 00010000:00010000:2.0:1789762545.126181:0:6664:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762545.126193:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: 0000000027c6d3a7/0x416193e012b00e0f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x44801000000000 nid: 0@lo remote: 0x416193e012b00e08 expref: 10 pid: 5799 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.126213:0:6664:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762545.126250:0:6664:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@000000009b01ba36 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:bbe4e1bd-8437-4448-83b9-0b3f7551a439+8:3274:x1876702022079360:12345-0@lo:103: Request processed in 82us (193us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.126257:0:6664:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 0 00000100:00100000:2.0:1789762545.126411:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022079488 00000100:00100000:2.0:1789762545.126416:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022079488 00000100:00100000:2.0:1789762545.126439:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 26 00000100:00100000:2.0:1789762545.126447:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@00000000bf43d849 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+8:6644:x1876702022079488:12345-0@lo:101: 00010000:00010000:2.0:1789762545.126454:0:5798:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762545.126460:0:5798:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: 00000000d165cc67/0x416193e012b00e2b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b00e24 expref: -99 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.126469:0:5798:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: 00000000d165cc67/0x416193e012b00e2b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b00e24 expref: 9 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.126474:0:5798:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: 00000000d165cc67/0x416193e012b00e2b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b00e24 expref: 9 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.126479:0:5798:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock 00000000d165cc67, rc 0) 00000100:00100000:2.0:1789762545.126506:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@00000000bf43d849 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+9:6644:x1876702022079488:12345-0@lo:101: Request processed in 61us (129us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.126519:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 26 00000100:00100000:2.0:1789762545.126910:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022079744 00000100:00100000:2.0:1789762545.126915:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022079744 00000100:00100000:2.0:1789762545.126940:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 28 00000100:00100000:2.0:1789762545.126946:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@00000000deb0e7ef pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+9:6644:x1876702022079744:12345-0@lo:503: 00000100:00100000:2.0:1789762545.127039:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@00000000deb0e7ef pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+9:6644:x1876702022079744:12345-0@lo:503: Request processed in 93us (161us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.127047:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 28 00000100:00100000:2.0:1789762545.127820:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022080000 00000100:00100000:2.0:1789762545.127825:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022080000 00000100:00100000:2.0:1789762545.127850:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30 00000100:00100000:2.0:1789762545.127855:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@000000000af2b51d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+9:6644:x1876702022080000:12345-0@lo:502: 00000100:00100000:2.0:1789762545.127970:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@000000000af2b51d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+9:6644:x1876702022080000:12345-0@lo:502: Request processed in 114us (183us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.127977:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30 00000100:00100000:2.0:1789762545.138785:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022080512 00000100:00100000:2.0:1789762545.138790:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022080512 00000100:00100000:2.0:1789762545.138815:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 34 00000100:00100000:2.0:1789762545.138821:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@0000000099df4ed0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+10:6644:x1876702022080512:12345-0@lo:502: 00000100:00100000:2.0:1789762545.138939:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@0000000099df4ed0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+10:6644:x1876702022080512:12345-0@lo:502: Request processed in 118us (190us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.138945:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 34 00000100:00100000:2.0:1789762545.140549:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022080768 00000100:00100000:2.0:1789762545.140554:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022080768 00000100:00100000:2.0:1789762545.140586:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 36 00000100:00100000:2.0:1789762545.140593:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+10:6644:x1876702022080768:12345-0@lo:101: 00010000:00010000:2.0:1789762545.140606:0:5798:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762545.140614:0:5798:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec04000/0x416193e012b00e55 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b00e4e expref: -99 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.140627:0:5798:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec04000/0x416193e012b00e55 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b00e4e expref: 11 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.140633:0:5798:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec04000/0x416193e012b00e55 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b00e4e expref: 11 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762545.140638:0:5798:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec04000, rc 0) 00000100:00100000:2.0:1789762545.140675:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+11:6644:x1876702022080768:12345-0@lo:101: Request processed in 82us (151us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.140681:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 36 00000100:00100000:2.0:1789762545.140833:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022080896 00000100:00100000:2.0:1789762545.140837:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022080896 00000100:00100000:2.0:1789762545.140857:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 37 00000100:00100000:2.0:1789762545.140861:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+11:6644:x1876702022080896:12345-0@lo:501: 00000100:00100000:2.0:1789762545.140971:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+11:6644:x1876702022080896:12345-0@lo:501: Request processed in 108us (167us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.140975:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 37 00000100:00100000:2.0:1789762545.141510:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022081152 00000100:00100000:2.0:1789762545.141514:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022081152 00000100:00100000:2.0:1789762545.141535:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 39 00000100:00100000:2.0:1789762545.141539:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+11:6644:x1876702022081152:12345-0@lo:502: 00000100:00100000:2.0:1789762545.141644:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+11:6644:x1876702022081152:12345-0@lo:502: Request processed in 103us (161us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762545.141649:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 39 00000100:00100000:2.0:1789762546.426951:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022082176 00000100:00100000:2.0:1789762546.426959:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022082176 00000100:00100000:2.0:1789762546.427007:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 46 00000100:00100000:2.0:1789762546.427017:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+12:6858:x1876702022082176:12345-0@lo:502: 00000100:00100000:2.0:1789762546.427338:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+12:6858:x1876702022082176:12345-0@lo:502: Request processed in 320us (435us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.427351:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 46 00000100:00100000:2.0:1789762546.428027:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022082432 00000100:00100000:2.0:1789762546.428033:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022082432 00000100:00100000:2.0:1789762546.428149:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 48 00000100:00100000:2.0:1789762546.428169:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:5802:x1876702022082432:12345-0@lo:256: 20000000:01000000:2.0:1789762546.428191:0:5798:0:(mgs_nids.c:914:mgs_get_ir_logs()) Reading IR log lustre-mdtir bufsize 16384. 20000000:01000000:2.0:1789762546.428214:0:5798:0:(mgs_nids.c:307:mgs_nidtbl_read()) fsname lustre, entry size 44, pages 4052/1/4/3. 20000000:01000000:2.0:1789762546.428220:0:5798:0:(mgs_nids.c:307:mgs_nidtbl_read()) fsname lustre, entry size 44, pages 4008/1/4/3. 20000000:01000000:2.0:1789762546.428224:0:5798:0:(mgs_nids.c:318:mgs_nidtbl_read()) Read IR logs lustre return with 88, version 4 00000100:00100000:2.0:1789762546.428423:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:5802:x1876702022082432:12345-0@lo:256: Request processed in 257us (432us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.428442:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 48 00000100:00100000:2.0:1789762546.428984:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022082688 00000100:00100000:2.0:1789762546.428991:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022082688 00000100:00100000:2.0:1789762546.429028:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 50 00000100:00100000:2.0:1789762546.429040:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85085850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022082688:12345-0@lo:501: 00000100:00100000:2.0:1789762546.429510:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85085850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022082688:12345-0@lo:501: Request processed in 470us (562us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.429526:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 50 00000100:00100000:2.0:1789762546.430675:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022082944 00000100:00100000:2.0:1789762546.430682:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022082944 00000100:00100000:2.0:1789762546.430720:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:2.0:1789762546.430729:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85086050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:6860:x1876702022082944:12345-0@lo:502: 00000100:00100000:2.0:1789762546.431016:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85086050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:6860:x1876702022082944:12345-0@lo:502: Request processed in 288us (389us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.431029:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000040:01000000:2.0:1789762546.447270:0:6860:0:(llog_osd.c:2246:llog_osd_get_cat_list()) cat list: disk size=0, read=32 00000040:01000000:2.0:1789762546.447288:0:6860:0:(llog_obd.c:197:llog_setup()) obd lustre-OST0000-osc-MDT0000 ctxt 2 is initialized 00000040:00080000:2.0:1789762546.447777:0:6860:0:(llog_osd.c:211:llog_osd_read_header()) not reading header from 0-byte log 00000040:00080000:2.0:1789762546.447940:0:6860:0:(llog_osd.c:211:llog_osd_read_header()) not reading header from 0-byte log 00000040:00080000:2.0:1789762546.448010:0:6860:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x2:0x0].1, 64 off8256 00000040:00080000:2.0:1789762546.448082:0:6860:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].1, 64 off8256 00000020:00000080:2.0:1789762546.449037:0:6860:0:(obd_config.c:826:class_setup()) finished setup of obd lustre-OST0000-osc-MDT0000 (uuid lustre-MDT0000-mdtlov_UUID) 00000020:00000080:2.0:1789762546.449052:0:6860:0:(obd_config.c:1472:class_process_config()) processing cmd: cf00d 00000004:01000000:2.0:1789762546.449088:0:6860:0:(lod_lov.c:128:lod_add_device()) osp:lustre-OST0000_UUID idx:0 gen:1 00000020:00000080:2.0:1789762546.449222:0:6860:0:(genops.c:1385:class_connect()) connect: client lustre-MDT0000-mdtlov_UUID, cookie 0x416193e012b00eb0 00000100:00080000:2.0:1789762546.449234:0:6860:0:(import.c:510:import_select_connection()) lustre-OST0000-osc-MDT0000: connect to NID 0@lo last attempt 0 00000100:00080000:2.0:1789762546.449243:0:6860:0:(import.c:624:import_select_connection()) lustre-OST0000-osc-MDT0000: import ffff981ebc39c000 using connection 192.168.203.148@tcp/0@lo 00000100:00100000:2.0:1789762546.449316:0:6860:0:(import.c:826:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff981e851bb800 x1876702022083072/t0(0) o8->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1789762546.449336:0:6860:0:(pinger.c:373:ptlrpc_pinger_add_import()) adding pingable import lustre-MDT0000-mdtlov_UUID->lustre-OST0000_UUID 00000020:01000000:2.0:1789762546.449433:0:6860:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1789762546.449442:0:6860:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1789762546.449445:0:6860:0:(obd_config.c:1563:class_process_config()) marker 12 (0x2) lustre-OST0000 add osc 00000020:01000000:2.0:1789762546.449453:0:6860:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1789762546.449460:0:6860:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1789762546.449464:0:6860:0:(obd_config.c:1563:class_process_config()) marker 14 (0x1) lustre sys.timeout 00000020:00000080:2.0:1789762546.449473:0:6860:0:(obd_config.c:1472:class_process_config()) processing cmd: cf009 00000020:00000080:2.0:1789762546.449480:0:6860:0:(obd_config.c:1537:class_process_config()) changing lustre timeout from 20 to 20 00000100:00100000:2.0:1789762546.449534:0:6860:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851baa00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:6860:1876702022083456:0@lo:502: 00000100:00100000:2.0:1789762546.449633:0:6860:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762546.449677:0:6860:0:(client.c:2715:ptlrpc_set_wait()) set ffff981d827b3a80 going to sleep for 11 seconds 00000100:00100000:2.0:1789762546.449732:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022083072 00000100:00100000:2.0:1789762546.449742:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022083072 00000100:00100000:2.0:1789762546.449784:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 0 00000100:00100000:2.0:1789762546.449798:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:0+-99:3270:x1876702022083072:12345-0@lo:8: 00010000:00080000:2.0:1789762546.449824:0:6647:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0000: connection from lustre-MDT0000-mdtlov_UUID@0@lo t0 exp 0000000000000000 cur 69 last 0 00000020:00000080:2.0:1789762546.449857:0:6647:0:(genops.c:1385:class_connect()) connect: client lustre-MDT0000-mdtlov_UUID, cookie 0x416193e012b00eb7 00002000:00100000:2.0:1789762546.449900:0:6647:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0000: cli lustre-MDT0000-mdtlov_UUID/ffff981ebf664800 ocd_connect_flags: 0xa044401443000026 ocd_version: 211396a ocd_grant: 0 ocd_index: 0 ocd_group 0 00002000:00080000:2.0:1789762546.449910:0:6647:0:(ofd_obd.c:139:ofd_parse_connect_data()) lustre-OST0000: Received MDS connection for group 0 00000001:00000004:2.0:1789762546.449916:0:6647:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294967296 00002000:00100000:2.0:1789762546.450001:0:6647:0:(ofd_obd.c:227:ofd_parse_connect_data()) lustre-OST0000: cli does not support OBD_CONNECT_CKSUM 00002000:00080000:2.0:1789762546.450170:0:6647:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00000100:00080000:2.0:1789762546.450201:0:6647:0:(import.c:65:import_set_state_nolock()) ffff981ebf666000 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789762546.450284:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3270:x1876702022083072:12345-0@lo:8: Request processed in 487us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.450296:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 0 00000100:00100000:2.0:1789762546.450333:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022083456 00000100:00100000:2.0:1789762546.450341:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022083456 00000100:00100000:2.0:1789762546.450386:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 53 00000100:00100000:2.0:1789762546.450407:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85087050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:6860:x1876702022083456:12345-0@lo:502: 00000100:00100000:2.0:1789762546.450884:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85087050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:6860:x1876702022083456:12345-0@lo:502: Request processed in 479us (1249us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.450899:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 53 00000100:00100000:2.0:1789762546.451532:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022083584 00000100:00100000:2.0:1789762546.451544:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022083584 00000100:00100000:2.0:1789762546.451628:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 54 00000100:00100000:2.0:1789762546.451640:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022083584:12345-0@lo:101: 00010000:00010000:2.0:1789762546.451649:0:5798:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762546.451668:0:5798:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec05400/0x416193e012b00ec5 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b00ebe expref: -99 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762546.451687:0:5798:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec05400/0x416193e012b00ec5 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b00ebe expref: 15 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762546.451697:0:5798:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec05400/0x416193e012b00ec5 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b00ebe expref: 15 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762546.451705:0:5798:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05400, rc 0) 00000100:00100000:2.0:1789762546.451757:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022083584:12345-0@lo:101: Request processed in 120us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.451763:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 54 00000100:00100000:2.0:1789762546.452759:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022083712 00000100:00100000:2.0:1789762546.452765:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022083712 00000100:00100000:2.0:1789762546.452815:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 55 00000100:00100000:2.0:1789762546.452825:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85087450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022083712:12345-0@lo:501: 00000100:00100000:2.0:1789762546.453139:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85087450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022083712:12345-0@lo:501: Request processed in 312us (445us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.453154:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 55 00000100:00100000:2.0:1789762546.453370:0:5802:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4973800 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022083712:0@lo:501: 00000100:00100000:2.0:1789762546.453437:0:5802:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851ba680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022083840:0@lo:503: 00000100:00100000:2.0:1789762546.453489:0:5802:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762546.453552:0:5802:0:(client.c:2715:ptlrpc_set_wait()) set ffff981d827b3780 going to sleep for 11 seconds 00000100:00100000:2.0:1789762546.453571:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022083840 00000100:00100000:2.0:1789762546.453576:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022083840 00000100:00100000:2.0:1789762546.453626:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 56 00000100:00100000:2.0:1789762546.453642:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85085c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022083840:12345-0@lo:503: 00000100:00100000:2.0:1789762546.454801:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85085c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022083840:12345-0@lo:503: Request processed in 1165us (1315us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.454810:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 56 00000100:00100000:2.0:1789762546.454889:0:5802:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851ba680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022083840:0@lo:503: 00000100:00100000:2.0:1789762546.455762:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022083968 00000100:00100000:2.0:1789762546.455770:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022083968 00000100:00100000:2.0:1789762546.455823:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57 00000100:00100000:2.0:1789762546.455841:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:6870:x1876702022083968:12345-0@lo:502: 00000100:00100000:2.0:1789762546.456189:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:6870:x1876702022083968:12345-0@lo:502: Request processed in 349us (464us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.456199:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57 00000100:00100000:2.0:1789762546.469040:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022084480 00000100:00100000:2.0:1789762546.469050:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022084480 00000100:00100000:2.0:1789762546.469154:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 0 00000100:00100000:2.0:1789762546.469168:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4972a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022084480:12345-0@lo:13: 00002000:00000024:2.0:1789762546.469197:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40960000 pending 0 free 3861905408 avail 3859808256 00000100:00100000:2.0:1789762546.469253:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4972a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022084480:12345-0@lo:13: Request processed in 89us (262us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.469269:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 0 00000100:00100000:2.0:1789762546.469302:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022084608 00000100:00100000:2.0:1789762546.469307:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022084608 00000100:00100000:2.0:1789762546.469341:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1 00000100:00100000:2.0:1789762546.469350:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:6868:x1876702022084608:12345-0@lo:7: 00002000:00080000:2.0:1789762546.474603:0:6649:0:(ofd_fs.c:432:ofd_seq_load()) lustre-OST0000: adding sequence 0x0 00002000:00080000:2.0:1789762546.474609:0:6649:0:(ofd_dev.c:1208:ofd_get_info_hdl()) lustre-OST0000: LAST FID is [0x100000000:0x0:0x0] 00000100:00100000:2.0:1789762546.474672:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:6868:x1876702022084608:12345-0@lo:7: Request processed in 5322us (5620us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762546.474687:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1 00000100:00100000:2.0:1789762546.474945:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022084736 00000100:00100000:2.0:1789762546.474953:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022084736 00000100:00100000:2.0:1789762546.474988:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2 00000100:00100000:2.0:1789762546.474995:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:6868:x1876702022084736:12345-0@lo:5: 00002000:00080000:2.0:1789762546.475016:0:6648:0:(ofd_dev.c:1640:ofd_create_hdl()) ofd_last_id() = 0 -> diff = 1 00002000:00080000:2.0:1789762546.475019:0:6648:0:(ofd_dev.c:1747:ofd_create_hdl()) lustre-OST0000: reserve 1 objects in group 0x0 at 1 00000100:00100000:2.0:1789762546.475270:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:6868:x1876702022084736:12345-0@lo:5: Request processed in 264us (372us total) trans 4294967297 rc 0/0 00000100:00100000:2.0:1789762546.475279:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2 00000100:00100000:2.0:1789762546.475424:0:6655:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022084864 00000100:00100000:2.0:1789762546.475431:0:6655:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022084864 00000100:00100000:2.0:1789762546.475460:0:6655:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 0 00000100:00100000:2.0:1789762546.475466:0:6655:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_seq00_00:lustre-MDT0000-mdtlov_UUID+7:6868:x1876702022084864:12345-0@lo:700: 00000100:00100000:2.0:1789762546.475499:0:6655:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851ba680 pname:cluuid:pid:xid:nid:opc:job ll_ost_seq00_00:lustre-MDT0000-lwp-OST0000_UUID:6655:1876702022084992:0@lo:700: 00000100:00100000:2.0:1789762546.475528:0:6655:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762546.475549:0:6655:0:(client.c:2715:ptlrpc_set_wait()) set ffff981d827b3000 going to sleep for 11 seconds 00000100:00100000:2.0:1789762546.475623:0:5812:0:(service.c:3462:ptlrpc_start_thread()) mdt_seqs[-1] started 2 min 2 max 256 00000100:00100000:2.0:1789762546.475629:0:5812:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'mdt_seqs_0002' 00000001:00080000:2.0:1789762546.520008:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967297 is committed 00000001:00100000:2.0:1789762546.520027:0:6674:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of lustre-MDT0000-mdtlov_UUID 00000004:02000040:2.0:1789762546.529821:0:6868:0:(osp_precreate.c:485:osp_precreate_rollover_new_seq()) lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 00000100:00100000:2.0:1789762546.532249:0:6868:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851bb100 pname:cluuid:pid:xid:nid:opc:job osp-pre-0-0:lustre-MDT0000-mdtlov_UUID:6868:1876702022085120:0@lo:5: 00000100:00100000:2.0:1789762546.532330:0:6868:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762546.532383:0:6868:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e99678540 going to sleep for 11 seconds 00010000:00080000:2.0:1789762546.539669:0:6868:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=92324) req@ffff981e851bb100 x1876702022085120/t0(0) o5->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 432/432 e 0 to 0 dl 1789762562 ref 2 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762546.539694:0:6868:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851bb100 pname:cluuid:pid:xid:nid:opc:job osp-pre-0-0:lustre-MDT0000-mdtlov_UUID:6868:1876702022085120:0@lo:5: 00000004:00080000:2.0:1789762546.539705:0:6868:0:(osp_precreate.c:705:osp_precreate_send()) lustre-OST0000-osc-MDT0000: current precreated pool: [0x240000400:0x1:0x0]-[0x240000400:0x21:0x0] 00000001:02000400:2.0:1789762547.208836:0:7020:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg348-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all 00010000:00010000:2.0:1789762549.687262:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762550.711185:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762551.506733:0:6662:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1789762551.506750:0:6662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022085504 00000100:00100000:2.0:1789762551.506753:0:6662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022085504 00000100:00100000:2.0:1789762551.506801:0:6662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5 00000100:00100000:2.0:1789762551.506813:0:6662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e93348850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+5:5798:x1876702022085504:12345-0@lo:104: 00010000:00010000:2.0:1789762551.506832:0:6662:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.148@tcp lock: ffff981e81682800/0x416193e012b00e94 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00e9b expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.506928:0:6662:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:2.0:1789762551.506943:0:6662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e93348850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+5:5798:x1876702022085504:12345-0@lo:104: Request processed in 130us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762551.506976:0:6662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5 00000100:00100000:2.0:1789762551.507035:0:6662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022085888 00000100:00100000:2.0:1789762551.507042:0:6662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022085888 00010000:00010000:2.0:1789762551.507074:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1789762551.507081:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.148@tcp lock: ffff981e81682800/0x416193e012b00e94 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00e9b expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507127:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81682800 already unused, calling callback (ffffffffc0592c20) 10000000:00010000:2.0:1789762551.507132:0:5793:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.148@tcp lock: ffff981e81682800/0x416193e012b00e94 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400400000000 nid: local remote: 0x416193e012b00e9b expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507162:0:5793:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.148@tcp lock: ffff981e81682800/0x416193e012b00e94 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1408400000000 nid: local remote: 0x416193e012b00e9b expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789762551.507181:0:5793:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.148@tcp lock: ffff981e81682800/0x416193e012b00e94 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1409400000000 nid: local remote: 0x416193e012b00e9b expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762551.507195:0:5793:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789762551.507246:0:5793:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.148@tcp lock: ffff981e81682800/0x416193e012b00e94 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00e9b expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507255:0:5793:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762551.507280:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.148@tcp lock: ffff981e81682800/0x416193e012b00e94 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00e9b expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507290:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.148@tcp lock: ffff981e81682800/0x416193e012b00e94 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00e9b expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507306:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1789762551.507313:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.148@tcp lock: ffff981e81683200/0x416193e012b00ecc lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00ed3 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507335:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81683200 already unused, calling callback (ffffffffc0592c20) 10000000:00010000:2.0:1789762551.507340:0:5793:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.148@tcp lock: ffff981e81683200/0x416193e012b00ecc lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400400000000 nid: local remote: 0x416193e012b00ed3 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507360:0:5793:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.148@tcp lock: ffff981e81683200/0x416193e012b00ecc lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1408400000000 nid: local remote: 0x416193e012b00ed3 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789762551.507381:0:5793:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.148@tcp lock: ffff981e81683200/0x416193e012b00ecc lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1409400000000 nid: local remote: 0x416193e012b00ed3 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762551.507399:0:5793:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789762551.507423:0:5793:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.148@tcp lock: ffff981e81683200/0x416193e012b00ecc lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00ed3 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507431:0:5793:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762551.507451:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.148@tcp lock: ffff981e81683200/0x416193e012b00ecc lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00ed3 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507478:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.148@tcp lock: ffff981e81683200/0x416193e012b00ecc lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00ed3 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507494:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762551.507501:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00e24 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00e2b expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507528:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfcc00 already unused, calling callback (ffffffffc0592c20) 10000000:00010000:2.0:1789762551.507536:0:5793:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00e24 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0x416193e012b00e2b expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507561:0:5793:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00e24 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0x416193e012b00e2b expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789762551.507573:0:5793:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00e24 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0x416193e012b00e2b expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762551.507590:0:5793:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789762551.507617:0:5793:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00e24 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00e2b expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507637:0:5793:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762551.507653:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00e24 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00e2b expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507665:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00e24 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00e2b expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507693:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762551.507697:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762551.507703:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762551.507736:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022086016:0@lo:103: 00000100:00100000:2.0:1789762551.507782:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762551.507806:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022086528:0@lo:103: 00000100:00100000:2.0:1789762551.507846:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762551.507890:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7 00000100:00100000:2.0:1789762551.507907:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:3273:x1876702022086400:12345-0@lo:103: 00010000:00010000:2.0:1789762551.507919:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762551.507930:0:5791:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff981e8ec04000/0x416193e012b00e55 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x416193e012b00e4e expref: 17 pid: 5798 timeout: 224 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507950:0:5791:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff981e8ec04000/0x416193e012b00e55 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x416193e012b00e4e expref: 17 pid: 5798 timeout: 224 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.507995:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff981e8ec04000/0x416193e012b00e55 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x416193e012b00e4e expref: 17 pid: 5798 timeout: 224 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.508025:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762551.508062:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:3273:x1876702022086400:12345-0@lo:103: Request processed in 160us (309us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762551.508078:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7 00000100:00100000:2.0:1789762551.508159:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4bcf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022086400:0@lo:103: 00000100:00100000:2.0:1789762551.508257:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4bcca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022086144:0@lo:103: 00000100:00100000:2.0:1789762551.520860:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022086656 00000100:00100000:2.0:1789762551.520866:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022086656 00000100:00100000:2.0:1789762551.520899:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 62 00000100:00100000:2.0:1789762551.520907:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85055850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+11:7588:x1876702022086656:12345-0@lo:101: 00010000:00010000:2.0:1789762551.520917:0:5797:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762551.520927:0:5797:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec04000/0x416193e012b00ef6 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b00eef expref: -99 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.520944:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec04000/0x416193e012b00ef6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b00eef expref: 12 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.520951:0:5797:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec04000/0x416193e012b00ef6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b00eef expref: 12 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.520957:0:5797:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec04000, rc 0) 00000100:00100000:2.0:1789762551.520989:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85055850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+12:7588:x1876702022086656:12345-0@lo:101: Request processed in 83us (207us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762551.520995:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 62 00000100:00100000:2.0:1789762551.521622:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022086912 00000100:00100000:2.0:1789762551.521629:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022086912 00000100:00100000:2.0:1789762551.521662:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 64 00000100:00100000:2.0:1789762551.521671:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+12:7588:x1876702022086912:12345-0@lo:101: 00010000:00010000:2.0:1789762551.521680:0:5797:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762551.521690:0:5797:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec05e00/0x416193e012b00f04 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b00efd expref: -99 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.521717:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec05e00/0x416193e012b00f04 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b00efd expref: 14 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.521729:0:5797:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec05e00/0x416193e012b00f04 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b00efd expref: 14 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.521743:0:5797:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05e00, rc 0) 00000100:00100000:2.0:1789762551.521785:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:7588:x1876702022086912:12345-0@lo:101: Request processed in 115us (242us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762551.521793:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 64 00000100:00100000:2.0:1789762551.522245:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022087296 00000100:00100000:2.0:1789762551.522249:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022087296 00000100:00100000:2.0:1789762551.522286:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 66 00000100:00100000:2.0:1789762551.522292:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+12:7588:x1876702022087296:12345-0@lo:503: 00000100:00100000:2.0:1789762551.522418:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+12:7588:x1876702022087296:12345-0@lo:503: Request processed in 126us (224us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762551.522424:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 66 00000100:00100000:2.0:1789762551.523403:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022087552 00000100:00100000:2.0:1789762551.523408:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022087552 00000100:00100000:2.0:1789762551.523429:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 68 00000100:00100000:2.0:1789762551.523435:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+12:7588:x1876702022087552:12345-0@lo:502: 00000100:00100000:2.0:1789762551.523525:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+12:7588:x1876702022087552:12345-0@lo:502: Request processed in 90us (153us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762551.523531:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 68 00000100:00100000:2.0:1789762551.536495:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022087808 00000100:00100000:2.0:1789762551.536502:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022087808 00000100:00100000:2.0:1789762551.536534:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 70 00000100:00100000:2.0:1789762551.536549:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850de050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:7588:x1876702022087808:12345-0@lo:501: 00000100:00100000:2.0:1789762551.536729:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850de050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:7588:x1876702022087808:12345-0@lo:501: Request processed in 179us (270us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762551.536740:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 70 00000100:00100000:2.0:1789762551.537538:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022088064 00000100:00100000:2.0:1789762551.537543:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022088064 00000100:00100000:2.0:1789762551.537576:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 72 00000100:00100000:2.0:1789762551.537585:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:7588:x1876702022088064:12345-0@lo:502: 00000100:00100000:2.0:1789762551.537730:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:7588:x1876702022088064:12345-0@lo:502: Request processed in 144us (227us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762551.537740:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 72 00000100:00100000:2.0:1789762551.539126:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022088320 00000100:00100000:2.0:1789762551.539133:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022088320 00000100:00100000:2.0:1789762551.539175:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 74 00000100:00100000:2.0:1789762551.539186:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850ddc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:7588:x1876702022088320:12345-0@lo:101: 00010000:00010000:2.0:1789762551.539194:0:5797:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762551.539206:0:5797:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec05000/0x416193e012b00f2e lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b00f27 expref: -99 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.539235:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec05000/0x416193e012b00f2e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b00f27 expref: 14 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.539246:0:5797:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec05000/0x416193e012b00f2e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b00f27 expref: 14 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.539255:0:5797:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05000, rc 0) 00000100:00100000:2.0:1789762551.539294:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850ddc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:7588:x1876702022088320:12345-0@lo:101: Request processed in 111us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762551.539312:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 74 00000100:00100000:2.0:1789762551.552901:0:6662:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022089728 00000100:00100000:2.0:1789762551.552908:0:6662:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022089728 00000100:00100000:2.0:1789762551.552949:0:6662:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10 00000100:00100000:2.0:1789762551.552961:0:6662:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+3:5820:x1876702022089728:12345-0@lo:104: 00010000:00010000:2.0:1789762551.552974:0:6662:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.148@tcp lock: ffff981e81683a00/0x416193e012b00e86 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00e8d expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762551.553059:0:6662:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1789762551.553067:0:6662:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+3:5820:x1876702022089728:12345-0@lo:104: Request processed in 109us (210us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762551.553093:0:6662:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10 00010000:00010000:2.0:1789762551.735189:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762551.927659:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022089984 00000100:00100000:2.0:1789762551.927672:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022089984 00000100:00100000:2.0:1789762551.927741:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1 00000100:00100000:2.0:1789762551.927757:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022089984:12345-0@lo:13: 00002000:00000024:2.0:1789762551.927797:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762551.927877:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022089984:12345-0@lo:13: Request processed in 125us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762551.927885:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1 00000100:00100000:2.0:1789762552.375434:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022090112 00000100:00100000:2.0:1789762552.375443:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022090112 00000100:00100000:2.0:1789762552.375499:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 80 00000100:00100000:2.0:1789762552.375512:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:5802:x1876702022090112:12345-0@lo:101: 00010000:00010000:2.0:1789762552.375526:0:5797:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762552.375540:0:5797:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec05600/0x416193e012b00f58 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b00f51 expref: -99 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762552.375574:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec05600/0x416193e012b00f58 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b00f51 expref: 14 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762552.375595:0:5797:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec05600/0x416193e012b00f58 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b00f51 expref: 14 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762552.375605:0:5797:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05600, rc 0) 00000100:00100000:2.0:1789762552.375661:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022090112:12345-0@lo:101: Request processed in 152us (315us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.375669:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 80 00000100:00100000:2.0:1789762552.376334:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022090368 00000100:00100000:2.0:1789762552.376354:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022090368 00000100:00100000:2.0:1789762552.376378:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 82 00000100:00100000:2.0:1789762552.376384:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022090368:12345-0@lo:503: 00000100:00100000:2.0:1789762552.376640:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022090368:12345-0@lo:503: Request processed in 256us (338us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.376646:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 82 00000100:00100000:2.0:1789762552.376991:0:7721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851bb480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:7721:1876702022090496:0@lo:502: 00000100:00100000:2.0:1789762552.377017:0:7721:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762552.377031:0:7721:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebf629b40 going to sleep for 11 seconds 00000100:00100000:2.0:1789762552.377472:0:7721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851bb480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:7721:1876702022090496:0@lo:502: 00000100:00100000:2.0:1789762552.377508:0:7721:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851bb480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:7721:1876702022090624:0@lo:502: 00000100:00100000:2.0:1789762552.377532:0:7721:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762552.377550:0:7721:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebf629b40 going to sleep for 11 seconds 00000100:00100000:2.0:1789762552.377790:0:7721:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851bb480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:7721:1876702022090624:0@lo:502: 00000040:00080000:2.0:1789762552.377817:0:7721:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 39 count 39 00000100:00100000:2.0:1789762552.378334:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022090880 00000100:00100000:2.0:1789762552.378339:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022090880 00000100:00100000:2.0:1789762552.378362:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 86 00000100:00100000:2.0:1789762552.378367:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850da050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022090880:12345-0@lo:501: 00000100:00100000:2.0:1789762552.378481:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850da050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022090880:12345-0@lo:501: Request processed in 113us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.378487:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 86 00000100:00100000:2.0:1789762552.379227:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022091136 00000100:00100000:2.0:1789762552.379231:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022091136 00000100:00100000:2.0:1789762552.379256:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 88 00000100:00100000:2.0:1789762552.379262:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:7722:x1876702022091136:12345-0@lo:502: 00000100:00100000:2.0:1789762552.379368:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:7722:x1876702022091136:12345-0@lo:502: Request processed in 106us (174us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.379373:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 88 00000100:00100000:2.0:1789762552.379549:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022091264 00000100:00100000:2.0:1789762552.379553:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022091264 00000100:00100000:2.0:1789762552.379575:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 89 00000100:00100000:2.0:1789762552.379579:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850db450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:7722:x1876702022091264:12345-0@lo:502: 00000100:00100000:2.0:1789762552.379662:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850db450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:7722:x1876702022091264:12345-0@lo:502: Request processed in 82us (144us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.379667:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 89 00000100:00100000:2.0:1789762552.379953:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022091392 00000100:00100000:2.0:1789762552.379958:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022091392 00000100:00100000:2.0:1789762552.379980:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 90 00000100:00100000:2.0:1789762552.379984:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022091392:12345-0@lo:101: 00010000:00010000:2.0:1789762552.379989:0:5797:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762552.380001:0:5797:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec05a00/0x416193e012b00f74 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b00f6d expref: -99 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762552.380013:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec05a00/0x416193e012b00f74 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b00f6d expref: 16 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762552.380020:0:5797:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec05a00/0x416193e012b00f74 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b00f6d expref: 16 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762552.380025:0:5797:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05a00, rc 0) 00000100:00100000:2.0:1789762552.380050:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:5802:x1876702022091392:12345-0@lo:101: Request processed in 66us (122us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.380054:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 90 00000100:00100000:2.0:1789762552.380181:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022091520 00000100:00100000:2.0:1789762552.380186:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022091520 00000100:00100000:2.0:1789762552.380220:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 91 00000100:00100000:2.0:1789762552.380226:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:5802:x1876702022091520:12345-0@lo:501: 00000100:00100000:2.0:1789762552.380324:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:5802:x1876702022091520:12345-0@lo:501: Request processed in 98us (161us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.380329:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 91 00000100:00100000:2.0:1789762552.380416:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022091648 00000100:00100000:2.0:1789762552.380420:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022091648 00000100:00100000:2.0:1789762552.380441:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 92 00000100:00100000:2.0:1789762552.380445:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:5802:x1876702022091648:12345-0@lo:503: 00000100:00100000:2.0:1789762552.380522:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:5802:x1876702022091648:12345-0@lo:503: Request processed in 76us (123us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.380527:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 92 00000100:00100000:2.0:1789762552.380706:0:7723:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851bb480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:7723:1876702022091776:0@lo:502: 00000100:00100000:2.0:1789762552.380737:0:7723:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762552.380752:0:7723:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebf6293c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762552.380987:0:7723:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851bb480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:7723:1876702022091776:0@lo:502: 00000100:00100000:2.0:1789762552.381020:0:7723:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851bb480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:7723:1876702022091904:0@lo:502: 00000100:00100000:2.0:1789762552.381043:0:7723:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762552.381056:0:7723:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebf6293c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762552.381264:0:7723:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851bb480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:7723:1876702022091904:0@lo:502: 00000040:00080000:2.0:1789762552.381286:0:7723:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 39 count 39 00000100:00100000:2.0:1789762552.381777:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022092160 00000100:00100000:2.0:1789762552.381781:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022092160 00000100:00100000:2.0:1789762552.381804:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 96 00000100:00100000:2.0:1789762552.381809:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:5802:x1876702022092160:12345-0@lo:256: 20000000:01000000:2.0:1789762552.381820:0:5797:0:(mgs_nids.c:914:mgs_get_ir_logs()) Reading IR log lustre-mdtir bufsize 16384. 20000000:01000000:2.0:1789762552.381834:0:5797:0:(mgs_nids.c:307:mgs_nidtbl_read()) fsname lustre, entry size 44, pages 4052/1/4/3. 20000000:01000000:2.0:1789762552.381836:0:5797:0:(mgs_nids.c:318:mgs_nidtbl_read()) Read IR logs lustre return with 44, version 5 00000100:00100000:2.0:1789762552.381913:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:5802:x1876702022092160:12345-0@lo:256: Request processed in 102us (193us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.381920:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 96 00000100:00100000:2.0:1789762552.382380:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022092416 00000100:00100000:2.0:1789762552.382385:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022092416 00000100:00100000:2.0:1789762552.382408:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 98 00000100:00100000:2.0:1789762552.382413:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:5802:x1876702022092416:12345-0@lo:501: 00000100:00100000:2.0:1789762552.382515:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:5802:x1876702022092416:12345-0@lo:501: Request processed in 101us (159us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.382522:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 98 00000100:00100000:2.0:1789762552.383146:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022092672 00000100:00100000:2.0:1789762552.383155:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022092672 00000100:00100000:2.0:1789762552.383177:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 100 00000100:00100000:2.0:1789762552.383182:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:7724:x1876702022092672:12345-0@lo:502: 00000100:00100000:2.0:1789762552.383335:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:7724:x1876702022092672:12345-0@lo:502: Request processed in 153us (219us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.383340:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 100 00000100:00100000:2.0:1789762552.385940:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022093056:0@lo:400: 00000100:00100000:2.0:1789762552.385998:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762552.386051:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022092800 00000100:00100000:2.0:1789762552.386056:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022092800 00000100:00100000:2.0:1789762552.386115:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 4 00000100:00100000:2.0:1789762552.386123:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0+-99:3270:x1876702022092800:12345-0@lo:8: 00010000:00080000:2.0:1789762552.386142:0:6881:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from lustre-MDT0000-mdtlov_UUID@0@lo t0 exp 0000000000000000 cur 75 last 0 00000020:00000080:2.0:1789762552.386172:0:6881:0:(genops.c:1385:class_connect()) connect: client lustre-MDT0000-mdtlov_UUID, cookie 0x416193e012b00fac 00002000:00100000:2.0:1789762552.386189:0:6881:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli lustre-MDT0000-mdtlov_UUID/ffff981ebfa2e000 ocd_connect_flags: 0xa044401443000026 ocd_version: 211396a ocd_grant: 0 ocd_index: 1 ocd_group 0 00002000:00080000:2.0:1789762552.386197:0:6881:0:(ofd_obd.c:139:ofd_parse_connect_data()) lustre-OST0001: Received MDS connection for group 0 00000001:00000004:2.0:1789762552.386202:0:6881:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967296 00002000:00100000:2.0:1789762552.386383:0:6881:0:(ofd_obd.c:227:ofd_parse_connect_data()) lustre-OST0001: cli does not support OBD_CONNECT_CKSUM 00002000:00080000:2.0:1789762552.386566:0:6881:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:2.0:1789762552.386596:0:6881:0:(import.c:65:import_set_state_nolock()) ffff981ebfa29000 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789762552.386644:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3270:x1876702022092800:12345-0@lo:8: Request processed in 522us (745us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.386653:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 4 00010000:00080000:2.0:1789762552.386720:0:3274:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff981e851ba300 x1876702022093056/t0(0) o400->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 224/224 e 0 to 0 dl 1789762568 ref 1 fl Rpc:RNQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762552.386733:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851ba300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022093056:0@lo:400: 00000100:00100000:2.0:1789762552.386788:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022093440 00000100:00100000:2.0:1789762552.386792:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022093440 00000100:00100000:2.0:1789762552.386825:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 102 00000100:00100000:2.0:1789762552.386836:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:5802:x1876702022093440:12345-0@lo:101: 00010000:00010000:2.0:1789762552.386845:0:5797:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762552.386858:0:5797:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec04600/0x416193e012b00fba lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b00fb3 expref: -99 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762552.386880:0:5797:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec04600/0x416193e012b00fba lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b00fb3 expref: 19 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762552.386891:0:5797:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec04600/0x416193e012b00fba lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b00fb3 expref: 19 pid: 5797 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762552.386900:0:5797:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec04600, rc 0) 00000100:00100000:2.0:1789762552.386937:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:5802:x1876702022093440:12345-0@lo:101: Request processed in 104us (168us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.386944:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 102 00000100:00100000:2.0:1789762552.387116:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022093568 00000100:00100000:2.0:1789762552.387122:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022093568 00000100:00100000:2.0:1789762552.387153:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 103 00000100:00100000:2.0:1789762552.387162:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:5802:x1876702022093568:12345-0@lo:501: 00000100:00100000:2.0:1789762552.387392:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:5802:x1876702022093568:12345-0@lo:501: Request processed in 231us (336us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789762552.387399:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 103 00000100:00100000:2.0:1789762552.387442:0:5802:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4bced80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022093568:0@lo:501: 10000000:01000000:2.0:1789762552.387468:0:5802:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.148@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:2.0:1789762552.387473:0:5802:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00fb3 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00fba expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762552.387488:0:5802:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00fb3 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b00fba expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762552.395396:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 1 committed 00000100:00100000:2.0:1789762552.395522:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022093696 00000100:00100000:2.0:1789762552.395527:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022093696 00000100:00100000:2.0:1789762552.395574:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2 00000100:00100000:2.0:1789762552.395594:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3271:x1876702022093696:12345-0@lo:13: 00002000:00000024:2.0:1789762552.395615:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40960000 pending 0 free 3861905408 avail 3859808256 00000100:00100000:2.0:1789762552.395663:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3271:x1876702022093696:12345-0@lo:13: Request processed in 79us (176us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.395668:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2 00010000:00080000:2.0:1789762552.395706:0:3271:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff981eb4bcdf80 x1876702022093696/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1789762568 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762552.395725:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4bcdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022093696:0@lo:13: 00000100:00100000:2.0:1789762552.399681:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022093952 00000100:00100000:2.0:1789762552.399687:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022093952 00000100:00100000:2.0:1789762552.399733:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6 00000100:00100000:2.0:1789762552.399744:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:7726:x1876702022093952:12345-0@lo:5: 00002000:00080000:2.0:1789762552.399760:0:6881:0:(ofd_dev.c:1640:ofd_create_hdl()) ofd_last_id() = 0 -> diff = 1 00002000:00080000:2.0:1789762552.399763:0:6881:0:(ofd_dev.c:1747:ofd_create_hdl()) lustre-OST0001: reserve 1 objects in group 0x0 at 1 00000100:00100000:2.0:1789762552.399966:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+7:7726:x1876702022093952:12345-0@lo:5: Request processed in 225us (319us total) trans 4294967297 rc 0/0 00000100:00100000:2.0:1789762552.399972:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6 00000100:00100000:2.0:1789762552.400292:0:5812:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022094208 00000100:00100000:2.0:1789762552.400296:0:5812:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022094208 00000100:00100000:2.0:1789762552.400333:0:5812:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1 00000100:00100000:2.0:1789762552.400338:0:5812:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_seqs_0001:lustre-MDT0000-lwp-OST0001_UUID+5:6656:x1876702022094208:12345-0@lo:700: 40000000:02000000:2.0:1789762552.409835:0:5812:0:(fid_handler.c:119:__seq_server_alloc_super()) ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost 00000100:00100000:2.0:1789762552.412547:0:5812:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_seqs_0001:lustre-MDT0000-lwp-OST0001_UUID+5:6656:x1876702022094208:12345-0@lo:700: Request processed in 12203us (12278us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762552.412558:0:5812:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1 00000001:00080000:2.0:1789762552.442579:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967297 is committed 00000001:00100000:2.0:1789762552.442593:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of lustre-MDT0000-mdtlov_UUID 00000100:00100000:2.0:1789762552.454000:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022094336 00000100:00100000:2.0:1789762552.454011:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022094336 00000100:00100000:2.0:1789762552.454068:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7 00000100:00100000:2.0:1789762552.454080:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:7726:x1876702022094336:12345-0@lo:5: 00002000:00080000:2.0:1789762552.459643:0:6648:0:(ofd_fs.c:432:ofd_seq_load()) lustre-OST0001: adding sequence 0x280000400 00002000:00080000:2.0:1789762552.459661:0:6648:0:(ofd_dev.c:1747:ofd_create_hdl()) lustre-OST0001: reserve 32 objects in group 0x280000400 at 2 00000100:00100000:2.0:1789762552.462378:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:7726:x1876702022094336:12345-0@lo:5: Request processed in 8297us (8427us total) trans 4294967298 rc 0/0 00000100:00100000:2.0:1789762552.462389:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7 00010000:00010000:2.0:1789762554.807218:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762556.199324:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014734464 00000100:00100000:2.0:1789762556.199348:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014734464 00000100:00100000:2.0:1789762556.200322:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014734592 00000100:00100000:2.0:1789762556.200327:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014734592 00000100:00100000:2.0:1789762556.200366:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 105 00000100:00100000:2.0:1789762556.200377:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+5:5581:x1876702014734592:12345-192.168.203.48@tcp:101: 00010000:00010000:2.0:1789762556.200386:0:5798:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762556.200400:0:5798:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec04400/0x416193e012b00fc8 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bce0c expref: -99 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762556.200431:0:5798:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec04400/0x416193e012b00fc8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce0c expref: 6 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762556.200441:0:5798:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec04400/0x416193e012b00fc8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce0c expref: 6 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762556.200448:0:5798:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec04400, rc 0) 00000100:00100000:2.0:1789762556.200478:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+6:5581:x1876702014734592:12345-192.168.203.48@tcp:101: Request processed in 104us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762556.200484:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 105 00000100:00100000:2.0:1789762556.202248:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762556.213931:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762556.222884:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762556.226466:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762557.235665:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014736512 00000100:00100000:2.0:1789762557.235671:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014736512 00000100:00100000:2.0:1789762557.235736:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 5 00000100:00100000:2.0:1789762557.235754:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:0:x1876702014736512:12345-192.168.203.48@tcp:41: 00000004:00000024:2.0:1789762557.235792:0:5806:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 3407872 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762557.235857:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:0:x1876702014736512:12345-192.168.203.48@tcp:41: Request processed in 109us (233us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762557.235872:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 5 00000100:00100000:2.0:1789762557.236746:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762557.237830:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014737024 00000100:00100000:2.0:1789762557.237836:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014737024 00000100:00100000:2.0:1789762557.237877:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 119 00000100:00100000:2.0:1789762557.237893:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85087450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+10:2366:x1876702014737024:12345-192.168.203.48@tcp:400: 00000100:00100000:2.0:1789762557.237948:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85087450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+10:2366:x1876702014737024:12345-192.168.203.48@tcp:400: Request processed in 59us (149us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762557.237961:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 119 00000100:00100000:2.0:1789762557.325985:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762557.559330:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022094976:0@lo:400: 00000100:00100000:2.0:1789762557.559390:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762557.559437:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022094592 00000100:00100000:2.0:1789762557.559444:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022094592 00000100:00100000:2.0:1789762557.559509:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 4 00000100:00100000:2.0:1789762557.559522:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4972300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022094592:12345-0@lo:13: 00002000:00000024:2.0:1789762557.559542:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762557.559595:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4972300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022094592:12345-0@lo:13: Request processed in 75us (258us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762557.559606:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 4 00000100:00100000:2.0:1789762557.559637:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022095104 00000100:00100000:2.0:1789762557.559643:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022095104 00000100:00100000:2.0:1789762557.559681:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8 00000100:00100000:2.0:1789762557.559690:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851ba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022094976:12345-0@lo:400: 00000100:00100000:2.0:1789762557.559733:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851ba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022094976:12345-0@lo:400: Request processed in 43us (343us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762557.559742:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8 00000100:00100000:2.0:1789762557.559783:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022094976:0@lo:400: 00010000:00010000:2.0:1789762557.879228:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762558.703397:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014737536 00000100:00100000:2.0:1789762558.703407:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014737536 00000100:00100000:2.0:1789762558.703468:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 9 00000100:00100000:2.0:1789762558.703476:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:5671:x1876702014737536:12345-192.168.203.48@tcp:41: 00000004:00000024:2.0:1789762558.703522:0:5806:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 3407872 pending 0 free 2261778432 avail 2259681280 00000100:00100000:2.0:1789762558.703580:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:5671:x1876702014737536:12345-192.168.203.48@tcp:41: Request processed in 105us (235us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762558.703587:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 9 00000100:00100000:2.0:1789762558.705000:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000040:00080000:2.0:1789762560.675208:0:8401:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 34 count 34 00000040:00080000:2.0:1789762560.676080:0:8402:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x9:0x0] index 8 count 8 00000040:00080000:2.0:1789762560.676930:0:8403:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 39 count 39 00000040:00080000:2.0:1789762560.677779:0:8404:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x8:0x0] index 11 count 11 00000100:00100000:2.0:1789762560.678845:0:7594:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022095872 00000100:00100000:2.0:1789762560.678853:0:7594:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022095872 00000100:00100000:2.0:1789762560.678904:0:7594:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13 00000100:00100000:2.0:1789762560.678919:0:7594:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf83b850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:bbe4e1bd-8437-4448-83b9-0b3f7551a439+5:8400:x1876702022095872:12345-0@lo:104: 00010000:00010000:2.0:1789762560.678938:0:7594:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.148@tcp lock: ffff981e87dfd000/0x416193e012b00f6d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00f74 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679063:0:7594:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:2.0:1789762560.679076:0:7594:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf83b850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_003:bbe4e1bd-8437-4448-83b9-0b3f7551a439+5:8400:x1876702022095872:12345-0@lo:104: Request processed in 159us (280us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762560.679163:0:7594:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13 00010000:00010000:2.0:1789762560.679245:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1789762560.679257:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00fb3 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00fba expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679301:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfcc00 already unused, calling callback (ffffffffc0592c20) 10000000:00010000:2.0:1789762560.679308:0:5793:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00fb3 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400400000000 nid: local remote: 0x416193e012b00fba expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679329:0:5793:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00fb3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1408400000000 nid: local remote: 0x416193e012b00fba expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789762560.679338:0:5793:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00fb3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1409400000000 nid: local remote: 0x416193e012b00fba expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762560.679345:0:5793:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789762560.679386:0:5793:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00fb3 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00fba expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679407:0:5793:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762560.679432:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00fb3 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00fba expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679492:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.148@tcp lock: ffff981e87dfcc00/0x416193e012b00fb3 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00fba expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679521:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:2.0:1789762560.679533:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.148@tcp lock: ffff981e87dfd000/0x416193e012b00f6d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00f74 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679545:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd000 already unused, calling callback (ffffffffc0592c20) 10000000:00010000:2.0:1789762560.679549:0:5793:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.148@tcp lock: ffff981e87dfd000/0x416193e012b00f6d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400400000000 nid: local remote: 0x416193e012b00f74 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679557:0:5793:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.148@tcp lock: ffff981e87dfd000/0x416193e012b00f6d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1408400000000 nid: local remote: 0x416193e012b00f74 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789762560.679565:0:5793:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.148@tcp lock: ffff981e87dfd000/0x416193e012b00f6d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1409400000000 nid: local remote: 0x416193e012b00f74 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762560.679610:0:5793:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789762560.679643:0:5793:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.148@tcp lock: ffff981e87dfd000/0x416193e012b00f6d lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00f74 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679652:0:5793:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762560.679664:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.148@tcp lock: ffff981e87dfd000/0x416193e012b00f6d lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00f74 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679722:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.148@tcp lock: ffff981e87dfd000/0x416193e012b00f6d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00f74 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679751:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762560.679758:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.148@tcp lock: ffff981e87dfc800/0x416193e012b00f51 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00f58 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679765:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc0592c20) 10000000:00010000:2.0:1789762560.679772:0:5793:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.148@tcp lock: ffff981e87dfc800/0x416193e012b00f51 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400400000000 nid: local remote: 0x416193e012b00f58 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679792:0:5793:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.148@tcp lock: ffff981e87dfc800/0x416193e012b00f51 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1408400000000 nid: local remote: 0x416193e012b00f58 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789762560.679798:0:5793:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.148@tcp lock: ffff981e87dfc800/0x416193e012b00f51 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1409400000000 nid: local remote: 0x416193e012b00f58 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762560.679805:0:5793:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789762560.679834:0:5793:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.148@tcp lock: ffff981e87dfc800/0x416193e012b00f51 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00f58 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679860:0:5793:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762560.679876:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.148@tcp lock: ffff981e87dfc800/0x416193e012b00f51 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00f58 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679906:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.148@tcp lock: ffff981e87dfc800/0x416193e012b00f51 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00f58 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.679944:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762560.679949:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762560.679956:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762560.679967:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762560.679983:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851bb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022096384:0@lo:103: 00000100:00100000:2.0:1789762560.680049:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762560.680141:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851bbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022097024:0@lo:103: 00000100:00100000:2.0:1789762560.680188:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762560.680230:0:3272:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789762560.680243:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4970e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022096896:0@lo:103: 00000100:00100000:2.0:1789762560.680291:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762560.680368:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4971500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022096640:0@lo:103: 00000100:00100000:2.0:1789762560.680426:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762560.680453:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdb8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022097152:0@lo:103: 00000100:00100000:2.0:1789762560.680513:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762560.680590:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12 00000100:00100000:2.0:1789762560.680618:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+24:3272:x1876702022097024:12345-0@lo:103: 00010000:00010000:2.0:1789762560.680635:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762560.680644:0:5791:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff981e8ec05600/0x416193e012b00f58 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x416193e012b00f51 expref: 24 pid: 5797 timeout: 233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.680669:0:5791:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff981e8ec05600/0x416193e012b00f58 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x416193e012b00f51 expref: 24 pid: 5797 timeout: 233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.680751:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff981e8ec05600/0x416193e012b00f58 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 14 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x416193e012b00f51 expref: 23 pid: 5797 timeout: 233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.680837:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762560.680893:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+21:3272:x1876702022097024:12345-0@lo:103: Request processed in 279us (700us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762560.680915:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12 00000100:00100000:2.0:1789762560.680969:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 16 00000100:00100000:2.0:1789762560.680996:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:3274:x1876702022097280:12345-0@lo:103: 00010000:00010000:2.0:1789762560.681020:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762560.681039:0:5791:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff981e8ec05e00/0x416193e012b00f04 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x416193e012b00efd expref: 17 pid: 5797 timeout: 233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.681067:0:5791:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff981e8ec05e00/0x416193e012b00f04 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x416193e012b00efd expref: 16 pid: 5797 timeout: 233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.681204:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff981e8ec05e00/0x416193e012b00f04 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x416193e012b00efd expref: 15 pid: 5797 timeout: 233 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.681224:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762560.681296:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:3274:x1876702022097280:12345-0@lo:103: Request processed in 315us (833us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762560.681301:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 16 00000100:00100000:2.0:1789762560.681328:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014737920 00000100:00100000:2.0:1789762560.681332:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014737920 00010000:00010000:2.0:1789762560.681351:0:5791:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff981e80031a00/0x416193e012b00fcf lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce13 expref: 10 pid: 5799 timeout: 183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762560.681381:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014738048 00000100:00100000:2.0:1789762560.681391:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014738048 00010000:00010000:2.0:1789762560.681409:0:5791:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff981e8ec04400/0x416193e012b00fc8 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce0c expref: 11 pid: 5798 timeout: 183 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762560.681447:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 19 00000100:00100000:2.0:1789762560.681466:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+11:2365:x1876702014738048:12345-192.168.203.48@tcp:103: 00010000:00010000:2.0:1789762560.681498:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762560.681509:0:5791:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff981e8ec04400/0x416193e012b00fc8 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce0c expref: 11 pid: 5798 timeout: 183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.681558:0:5791:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff981e8ec04400/0x416193e012b00fc8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce0c expref: 11 pid: 5798 timeout: 183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.681601:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff981e8ec04400/0x416193e012b00fc8 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x44801400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce0c expref: 10 pid: 5798 timeout: 183 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762560.681650:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762560.681710:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+9:2365:x1876702014738048:12345-192.168.203.48@tcp:103: Request processed in 242us (442us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762560.681733:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 19 00010000:00010000:2.0:1789762560.951281:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:2.0:1789762561.680719:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000001:00100000:2.0:1789762561.681562:0:5826:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000100:00080000:2.0:1789762562.167937:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762562.658854:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014738176 00000100:00100000:2.0:1789762562.658861:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014738176 00000100:00100000:2.0:1789762562.658926:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 121 00000100:00100000:2.0:1789762562.658949:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+8:2366:x1876702014738176:12345-192.168.203.48@tcp:400: 00000100:00100000:2.0:1789762562.659055:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+8:2366:x1876702014738176:12345-192.168.203.48@tcp:400: Request processed in 87us (284us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.659098:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 121 00000100:00100000:2.0:1789762562.679396:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdbad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022097792:0@lo:400: 00000100:00100000:2.0:1789762562.679469:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762562.679541:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdb9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022098304:0@lo:400: 00000100:00100000:2.0:1789762562.679585:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762562.679691:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11 00000100:00100000:2.0:1789762562.679721:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022097920:12345-0@lo:400: 00000100:00100000:2.0:1789762562.679862:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022097920:12345-0@lo:400: Request processed in 152us (370us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.679875:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11 00000100:00100000:2.0:1789762562.679937:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022097664 00000100:00100000:2.0:1789762562.679949:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022097664 00000100:00100000:2.0:1789762562.680025:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 8 00000100:00100000:2.0:1789762562.680071:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e90598a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022097664:12345-0@lo:13: 00002000:00000024:2.0:1789762562.680125:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762562.680301:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e90598a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022097664:12345-0@lo:13: Request processed in 249us (828us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.680321:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 8 00000100:00100000:2.0:1789762562.680418:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022097920:0@lo:400: 00000100:00100000:2.0:1789762562.681180:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022098560 00000100:00100000:2.0:1789762562.681185:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022098560 00000100:00100000:2.0:1789762562.681218:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 125 00000100:00100000:2.0:1789762562.681244:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85056450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:5802:x1876702022098560:12345-0@lo:503: 00000100:00100000:2.0:1789762562.681439:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85056450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:5802:x1876702022098560:12345-0@lo:503: Request processed in 213us (400us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.681485:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 125 00000100:00100000:2.0:1789762562.682717:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022098816 00000100:00100000:2.0:1789762562.682726:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022098816 00000100:00100000:2.0:1789762562.682801:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:2.0:1789762562.682812:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85057c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:8422:x1876702022098816:12345-0@lo:502: 00000100:00100000:2.0:1789762562.683049:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85057c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:8422:x1876702022098816:12345-0@lo:502: Request processed in 236us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.683079:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00000100:00100000:2.0:1789762562.683561:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022098944 00000100:00100000:2.0:1789762562.683566:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022098944 00000100:00100000:2.0:1789762562.683591:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 128 00000100:00100000:2.0:1789762562.683597:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+13:5802:x1876702022098944:12345-0@lo:101: 00010000:00010000:2.0:1789762562.683603:0:5798:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762562.683611:0:5798:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec04400/0x416193e012b01023 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b0101c expref: -99 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.683644:0:5798:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec04400/0x416193e012b01023 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b0101c expref: 14 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.683657:0:5798:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec04400/0x416193e012b01023 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b0101c expref: 14 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.683666:0:5798:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec04400, rc 0) 00000100:00100000:2.0:1789762562.683719:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022098944:12345-0@lo:101: Request processed in 120us (205us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.683728:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 128 00000100:00100000:2.0:1789762562.683952:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022099072 00000100:00100000:2.0:1789762562.683959:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022099072 00000100:00100000:2.0:1789762562.684006:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1789762562.684013:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022099072:12345-0@lo:501: 00000100:00100000:2.0:1789762562.684349:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022099072:12345-0@lo:501: Request processed in 334us (464us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.684359:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1789762562.684569:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022099200 00000100:00100000:2.0:1789762562.684577:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022099200 00000100:00100000:2.0:1789762562.684612:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:2.0:1789762562.684620:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022099200:12345-0@lo:503: 00000100:00100000:2.0:1789762562.684800:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022099200:12345-0@lo:503: Request processed in 177us (293us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.684814:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00100000:2.0:1789762562.685432:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022099328 00000100:00100000:2.0:1789762562.685437:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022099328 00000100:00100000:2.0:1789762562.685462:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:2.0:1789762562.685467:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850de450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:8423:x1876702022099328:12345-0@lo:502: 00000100:00100000:2.0:1789762562.685584:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850de450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:8423:x1876702022099328:12345-0@lo:502: Request processed in 116us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.685590:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00100000:2.0:1789762562.685835:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022099456 00000100:00100000:2.0:1789762562.685840:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022099456 00000100:00100000:2.0:1789762562.685863:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 132 00000100:00100000:2.0:1789762562.685868:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:8423:x1876702022099456:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762562.686053:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:8423:x1876702022099456:12345-0@lo:502:llog_process_th.0 Request processed in 181us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.686063:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 132 00000100:00100000:2.0:1789762562.686560:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022099584 00000100:00100000:2.0:1789762562.686577:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022099584 00000100:00100000:2.0:1789762562.686615:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 133 00000100:00100000:2.0:1789762562.686622:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+14:5802:x1876702022099584:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789762562.686632:0:5799:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762562.686661:0:5799:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec05e00/0x416193e012b01031 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b0102a expref: -99 pid: 5799 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.686681:0:5799:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec05e00/0x416193e012b01031 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b0102a expref: 15 pid: 5799 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.686694:0:5799:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec05e00/0x416193e012b01031 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b0102a expref: 15 pid: 5799 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.686704:0:5799:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05e00, rc 0) 00000100:00100000:2.0:1789762562.686760:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022099584:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 137us (277us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.686769:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 133 00000100:00100000:2.0:1789762562.687016:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022099712 00000100:00100000:2.0:1789762562.687023:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022099712 00000100:00100000:2.0:1789762562.687073:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 134 00000100:00100000:2.0:1789762562.687081:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850de050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022099712:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789762562.687339:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850de050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022099712:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 255us (399us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.687356:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 134 00000100:00100000:2.0:1789762562.687493:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022099840 00000100:00100000:2.0:1789762562.687498:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022099840 00000100:00100000:2.0:1789762562.687521:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1789762562.687525:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022099840:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789762562.687634:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022099840:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 107us (179us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.687641:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1789762562.688252:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022099968 00000100:00100000:2.0:1789762562.688265:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022099968 00000100:00100000:2.0:1789762562.688326:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 136 00000100:00100000:2.0:1789762562.688346:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:8424:x1876702022099968:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762562.688502:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:8424:x1876702022099968:12345-0@lo:502:llog_process_th.0 Request processed in 165us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.688511:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 136 00000100:00100000:2.0:1789762562.688635:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022100096 00000100:00100000:2.0:1789762562.688640:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022100096 00000100:00100000:2.0:1789762562.688697:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 137 00000100:00100000:2.0:1789762562.688706:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850df450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:8424:x1876702022100096:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762562.688887:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850df450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:8424:x1876702022100096:12345-0@lo:502:llog_process_th.0 Request processed in 181us (288us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.688894:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000100:00100000:2.0:1789762562.689415:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022100224 00000100:00100000:2.0:1789762562.689420:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022100224 00000100:00100000:2.0:1789762562.689466:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:2.0:1789762562.689474:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850ddc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+15:5802:x1876702022100224:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789762562.689485:0:5798:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762562.689499:0:5798:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec05600/0x416193e012b0103f lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b01038 expref: -99 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.689522:0:5798:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec05600/0x416193e012b0103f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b01038 expref: 16 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.689545:0:5798:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec05600/0x416193e012b0103f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b01038 expref: 16 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.689554:0:5798:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05600, rc 0) 00000100:00100000:2.0:1789762562.689604:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850ddc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:5802:x1876702022100224:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 130us (264us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.689612:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:2.0:1789762562.689841:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022100352 00000100:00100000:2.0:1789762562.689846:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022100352 00000100:00100000:2.0:1789762562.689868:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 139 00000100:00100000:2.0:1789762562.689873:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:5802:x1876702022100352:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789762562.690169:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:5802:x1876702022100352:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 287us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.690199:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 139 00000100:00100000:2.0:1789762562.690354:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022100480 00000100:00100000:2.0:1789762562.690362:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022100480 00000100:00100000:2.0:1789762562.690405:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 140 00000100:00100000:2.0:1789762562.690410:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850da850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:5802:x1876702022100480:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789762562.690520:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850da850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:5802:x1876702022100480:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 109us (220us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.690526:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 140 00000100:00100000:2.0:1789762562.690798:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022100608 00000100:00100000:2.0:1789762562.690803:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022100608 00000100:00100000:2.0:1789762562.690825:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 141 00000100:00100000:2.0:1789762562.690830:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:8425:x1876702022100608:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762562.690943:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:8425:x1876702022100608:12345-0@lo:502:llog_process_th.0 Request processed in 111us (182us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.690955:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 141 00000100:00100000:2.0:1789762562.691167:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022100736 00000100:00100000:2.0:1789762562.691176:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022100736 00000100:00100000:2.0:1789762562.691267:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:2.0:1789762562.691286:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:8425:x1876702022100736:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762562.691472:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:8425:x1876702022100736:12345-0@lo:502:llog_process_th.0 Request processed in 192us (406us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.691482:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:2.0:1789762562.691988:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022100864 00000100:00100000:2.0:1789762562.692025:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022100864 00000100:00100000:2.0:1789762562.692075:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 143 00000100:00100000:2.0:1789762562.692161:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850db850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+16:5802:x1876702022100864:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789762562.692197:0:5799:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762562.692214:0:5799:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec05000/0x416193e012b0104d lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b01046 expref: -99 pid: 5799 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.692243:0:5799:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec05000/0x416193e012b0104d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b01046 expref: 17 pid: 5799 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.692263:0:5799:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec05000/0x416193e012b0104d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b01046 expref: 17 pid: 5799 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.692277:0:5799:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05000, rc 0) 00000100:00100000:2.0:1789762562.692349:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850db850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:5802:x1876702022100864:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 261us (488us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.692361:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 143 00000100:00100000:2.0:1789762562.692521:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022100992 00000100:00100000:2.0:1789762562.692532:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022100992 00000100:00100000:2.0:1789762562.692558:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:2.0:1789762562.692564:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:5802:x1876702022100992:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789762562.692736:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:5802:x1876702022100992:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 169us (251us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.692749:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00100000:2.0:1789762562.692899:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022101120 00000100:00100000:2.0:1789762562.692903:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022101120 00000100:00100000:2.0:1789762562.692925:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 145 00000100:00100000:2.0:1789762562.692930:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:5802:x1876702022101120:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789762562.693185:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:5802:x1876702022101120:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 251us (319us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.693193:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00000100:00100000:2.0:1789762562.693603:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022101248 00000100:00100000:2.0:1789762562.693610:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022101248 00000100:00100000:2.0:1789762562.693646:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:2.0:1789762562.693657:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:8426:x1876702022101248:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762562.693862:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:8426:x1876702022101248:12345-0@lo:502:llog_process_th.0 Request processed in 209us (323us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.693870:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00100000:2.0:1789762562.694084:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022101376 00000100:00100000:2.0:1789762562.694119:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022101376 00000100:00100000:2.0:1789762562.694208:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 147 00000100:00100000:2.0:1789762562.694220:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850da450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:8426:x1876702022101376:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762562.694440:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850da450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:8426:x1876702022101376:12345-0@lo:502:llog_process_th.0 Request processed in 222us (410us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.694486:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 147 00000100:00100000:2.0:1789762562.694961:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022101504 00000100:00100000:2.0:1789762562.694968:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022101504 00000100:00100000:2.0:1789762562.695004:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:2.0:1789762562.695011:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:5802:x1876702022101504:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789762562.695021:0:5798:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762562.695033:0:5798:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec04600/0x416193e012b0105b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b01054 expref: -99 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.695052:0:5798:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec04600/0x416193e012b0105b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b01054 expref: 18 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.695063:0:5798:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec04600/0x416193e012b0105b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b01054 expref: 18 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.695071:0:5798:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec04600, rc 0) 00000100:00100000:2.0:1789762562.695179:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:5802:x1876702022101504:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 165us (309us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.695188:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00100000:2.0:1789762562.695391:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022101632 00000100:00100000:2.0:1789762562.695396:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022101632 00000100:00100000:2.0:1789762562.695420:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:2.0:1789762562.695425:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dbc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:5802:x1876702022101632:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789762562.695543:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dbc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:5802:x1876702022101632:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 117us (225us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.695549:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00100000:2.0:1789762562.695686:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022101760 00000100:00100000:2.0:1789762562.695702:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022101760 00000100:00100000:2.0:1789762562.695741:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:2.0:1789762562.695749:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850da050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:5802:x1876702022101760:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789762562.695900:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850da050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:5802:x1876702022101760:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 152us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.695906:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:2.0:1789762562.696361:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022101888 00000100:00100000:2.0:1789762562.696383:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022101888 00000100:00100000:2.0:1789762562.696408:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:2.0:1789762562.696413:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850db050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:8427:x1876702022101888:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762562.696613:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850db050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:8427:x1876702022101888:12345-0@lo:502:llog_process_th.0 Request processed in 197us (315us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.696622:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00100000:2.0:1789762562.696851:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022102016 00000100:00100000:2.0:1789762562.696856:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022102016 00000100:00100000:2.0:1789762562.696879:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1789762562.696901:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:8427:x1876702022102016:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762562.697078:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:8427:x1876702022102016:12345-0@lo:502:llog_process_th.0 Request processed in 191us (290us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.697123:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1789762562.697544:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022102144 00000100:00100000:2.0:1789762562.697549:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022102144 00000100:00100000:2.0:1789762562.697572:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1789762562.697578:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850db450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:5802:x1876702022102144:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789762562.697585:0:5799:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762562.697592:0:5799:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec05a00/0x416193e012b01069 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: local remote: 0x416193e012b01062 expref: -99 pid: 5799 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.697616:0:5799:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec05a00/0x416193e012b01069 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x416193e012b01062 expref: 19 pid: 5799 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.697624:0:5799:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec05a00/0x416193e012b01069 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x416193e012b01062 expref: 19 pid: 5799 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762562.697630:0:5799:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05a00, rc 0) 00000100:00100000:2.0:1789762562.697663:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850db450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:5802:x1876702022102144:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 85us (163us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762562.697668:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1789762562.697857:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022102272 00000100:00100000:2.0:1789762562.697861:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022102272 00000100:00100000:2.0:1789762562.697882:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:2.0:1789762562.697888:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:5802:x1876702022102272:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789762562.698068:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:5802:x1876702022102272:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 177us (277us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789762562.698080:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:2.0:1789762563.106466:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762563.107798:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014738816 00000100:00100000:2.0:1789762563.107806:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014738816 00000100:00100000:2.0:1789762563.107849:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 156 00000100:00100000:2.0:1789762563.107861:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+9:5595:x1876702014738816:12345-192.168.203.48@tcp:501: 00000100:00100000:2.0:1789762563.108232:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+9:5595:x1876702014738816:12345-192.168.203.48@tcp:501: Request processed in 371us (497us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762563.108242:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 156 00000100:00100000:2.0:1789762563.110862:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014739072 00000100:00100000:2.0:1789762563.110876:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014739072 00000100:00100000:2.0:1789762563.110914:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 158 00000100:00100000:2.0:1789762563.110926:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+9:6072:x1876702014739072:12345-192.168.203.48@tcp:502: 00000100:00100000:2.0:1789762563.111247:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+9:6072:x1876702014739072:12345-192.168.203.48@tcp:502: Request processed in 322us (448us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762563.111258:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 158 00000100:00100000:2.0:1789762563.113521:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014739328 00000100:00100000:2.0:1789762563.113529:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014739328 00000100:00100000:2.0:1789762563.113574:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 160 00000100:00100000:2.0:1789762563.113587:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+9:5595:x1876702014739328:12345-192.168.203.48@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789762563.113599:0:5798:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762563.113612:0:5798:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff981e8ec04000/0x416193e012b01077 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bce7c expref: -99 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762563.113635:0:5798:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e8ec04000/0x416193e012b01077 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce7c expref: 10 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762563.113650:0:5798:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff981e8ec04000/0x416193e012b01077 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce7c expref: 10 pid: 5798 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762563.113660:0:5798:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec04000, rc 0) 00000100:00100000:2.0:1789762563.113702:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+10:5595:x1876702014739328:12345-192.168.203.48@tcp:101:ll_cfg_requeue.0 Request processed in 118us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762563.113723:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 160 00010000:00010000:2.0:1789762564.023183:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789762564.052397:0:8465:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 4 count 4 00000100:00100000:2.0:1789762564.055599:0:5789:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022102528 00000100:00100000:2.0:1789762564.055608:0:5789:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022102528 00000100:00100000:2.0:1789762564.055659:0:5789:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022102656 00000100:00100000:2.0:1789762564.055664:0:5789:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022102656 00000100:00100000:2.0:1789762564.055748:0:5789:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1789762564.055763:0:5789:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022102784 00000100:00100000:2.0:1789762564.055766:0:5789:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022102784 00000100:00100000:2.0:1789762564.055790:0:5789:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 20 00000100:00100000:2.0:1789762564.055802:0:5789:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc68e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+5:8464:x1876702022102784:12345-0@lo:104: 00010000:00010000:2.0:1789762564.055819:0:5789:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.148@tcp lock: ffff981e81682400/0x416193e012b00e40 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00e47 expref: -99 pid: 6644 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762564.055874:0:5789:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1789762564.055881:0:5789:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc68e050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+4:8464:x1876702022102784:12345-0@lo:104: Request processed in 84us (167us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762564.055898:0:5789:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 20 00010000:00010000:2.0:1789762564.055950:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762564.056029:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762564.056036:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.148@tcp lock: ffff981e81683000/0x416193e012b00f19 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b00f20 expref: -99 pid: 7588 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762564.056058:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81683000 already unused, calling callback (ffffffffc0592c20) 10000000:00010000:2.0:1789762564.056065:0:6663:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.148@tcp lock: ffff981e81683000/0x416193e012b00f19 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0x416193e012b00f20 expref: -99 pid: 7588 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762564.056106:0:6663:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.148@tcp lock: ffff981e81683000/0x416193e012b00f19 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x416193e012b00f20 expref: -99 pid: 7588 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789762564.056114:0:6663:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.148@tcp lock: ffff981e81683000/0x416193e012b00f19 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x416193e012b00f20 expref: -99 pid: 7588 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762564.056123:0:6663:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 00010000:00010000:2.0:1789762564.056166:0:6663:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.148@tcp lock: ffff981e81683000/0x416193e012b00f19 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00f20 expref: -99 pid: 7588 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762564.056171:0:6663:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789762564.056182:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.148@tcp lock: ffff981e81683000/0x416193e012b00f19 lrc: 1/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00f20 expref: -99 pid: 7588 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762564.056198:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.148@tcp lock: ffff981e81683000/0x416193e012b00f19 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x416193e012b00f20 expref: -99 pid: 7588 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762564.056215:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762564.056219:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762564.056226:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762564.056271:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022102912 00000100:00100000:2.0:1789762564.056276:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022102912 00010000:00010000:2.0:1789762564.056305:0:5790:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff981e8ec05400/0x416193e012b00ec5 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x416193e012b00ebe expref: 21 pid: 5798 timeout: 237 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762564.056343:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 20 00000100:00100000:2.0:1789762564.056351:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+21:3274:x1876702022102912:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762564.056361:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762564.056367:0:5790:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff981e8ec05400/0x416193e012b00ec5 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x416193e012b00ebe expref: 21 pid: 5798 timeout: 237 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762564.056389:0:5790:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff981e8ec05400/0x416193e012b00ec5 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x416193e012b00ebe expref: 21 pid: 5798 timeout: 237 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762564.056412:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff981e8ec05400/0x416193e012b00ec5 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 9 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x416193e012b00ebe expref: 21 pid: 5798 timeout: 237 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762564.056445:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762564.056489:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:3274:x1876702022102912:12345-0@lo:103:ldlm_bl.0 Request processed in 138us (299us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762564.056495:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 20 00000100:00100000:2.0:1789762564.056588:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e90599500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022103040:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789762564.146192:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762564.149880:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762565.240789:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022103424 00000100:00100000:2.0:1789762565.240795:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022103424 00000100:00100000:2.0:1789762565.240824:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:2.0:1789762565.240832:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:5802:x1876702022103424:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789762565.241034:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:5802:x1876702022103424:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 201us (275us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762565.241043:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:2.0:1789762565.241762:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022103680 00000100:00100000:2.0:1789762565.241767:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022103680 00000100:00100000:2.0:1789762565.241794:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:2.0:1789762565.241801:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:8600:x1876702022103680:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762565.241920:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+17:8600:x1876702022103680:12345-0@lo:502:llog_process_th.0 Request processed in 120us (184us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762565.241926:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00100000:2.0:1789762565.244383:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022104192 00000100:00100000:2.0:1789762565.244388:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022104192 00000100:00100000:2.0:1789762565.244416:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:2.0:1789762565.244422:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:5802:x1876702022104192:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789762565.244529:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:5802:x1876702022104192:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 105us (168us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762565.244554:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00100000:2.0:1789762565.244780:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022104320 00000100:00100000:2.0:1789762565.244786:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022104320 00000100:00100000:2.0:1789762565.244813:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:2.0:1789762565.244819:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85086c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:8603:x1876702022104320:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762565.244933:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85086c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:8603:x1876702022104320:12345-0@lo:502:llog_process_th.0 Request processed in 115us (173us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762565.244939:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:2.0:1789762565.245383:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022104448 00000100:00100000:2.0:1789762565.245387:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022104448 00000100:00100000:2.0:1789762565.245414:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:2.0:1789762565.245420:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:8603:x1876702022104448:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762565.245556:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:8603:x1876702022104448:12345-0@lo:502:llog_process_th.0 Request processed in 136us (493us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762565.245563:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000020:01000000:2.0:1789762565.245646:0:5802:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 4-6 (rc=0) 10000000:01000000:2.0:1789762565.245664:0:5802:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.148@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:2.0:1789762565.245670:0:5802:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.148@tcp lock: ffff981e80031000/0x416193e012b0109a lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b010a1 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762565.245677:0:5802:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.148@tcp lock: ffff981e80031000/0x416193e012b0109a lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b010a1 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762565.245684:0:5802:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1789762565.245686:0:5802:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff981e903ad000 from 4 10000000:01000000:2.0:1789762565.245688:0:5802:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1789762565.245705:0:5802:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.148@tcp lock: ffff981e8ec04e00/0x416193e012b010a8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762565.245716:0:5802:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.148@tcp lock: ffff981e8ec04e00/0x416193e012b010a8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762565.245721:0:5802:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.148@tcp lock: ffff981e8ec04e00/0x416193e012b010a8 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762565.245733:0:5802:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022104576:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1789762565.245762:0:5802:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762565.245792:0:5802:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebf629540 going to sleep for 11 seconds 00000100:00100000:2.0:1789762565.245913:0:5802:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022104576:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789762565.245927:0:5802:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.148@tcp lock: ffff981e8ec04e00/0x416193e012b010a8 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b010af expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762565.245934:0:5802:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.148@tcp lock: ffff981e8ec04e00/0x416193e012b010a8 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x416193e012b010af expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762565.245971:0:5802:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022104704:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789762565.245998:0:5802:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762565.246015:0:5802:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebf629540 going to sleep for 11 seconds 00000100:00100000:2.0:1789762565.246326:0:5802:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022104704:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789762565.246361:0:5802:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022104832:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789762565.246405:0:5802:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762565.246442:0:5802:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebf629540 going to sleep for 11 seconds 00000100:00100000:2.0:1789762565.246669:0:5802:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022104832:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789762565.246896:0:8605:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:8605:1876702022104960:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762565.246955:0:8605:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762565.246977:0:8605:0:(client.c:2715:ptlrpc_set_wait()) set ffff981eaa97e780 going to sleep for 11 seconds 00000100:00100000:2.0:1789762565.247536:0:8605:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:8605:1876702022104960:0@lo:502:llog_process_th.0 00000020:01000000:2.0:1789762565.247568:0:8605:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1789762565.247576:0:8605:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1789762565.247578:0:8605:0:(obd_config.c:1563:class_process_config()) marker 3 (0x1) general lod.*.mdt_hash 00000020:01000000:2.0:1789762565.247589:0:8605:0:(obd_config.c:2004:class_config_llog_handler()) cmd ce032, instance name: general-ffff981e903ad000 00000020:00000080:2.0:1789762565.247593:0:8605:0:(obd_config.c:1472:class_process_config()) processing cmd: ce032 00000100:00100000:2.0:1789762565.247662:0:8605:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:8605:1876702022105088:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789762565.247699:0:8605:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762565.247730:0:8605:0:(client.c:2715:ptlrpc_set_wait()) set ffff981eaa97e780 going to sleep for 11 seconds 00000100:00100000:2.0:1789762565.248015:0:8605:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:bbe4e1bd-8437-4448-83b9-0b3f7551a439:8605:1876702022105088:0@lo:502:llog_process_th.0 00000020:01000000:2.0:1789762565.248041:0:8605:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1789762565.248045:0:8605:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1789762565.248047:0:8605:0:(obd_config.c:1563:class_process_config()) marker 3 (0x2) general lod.*.mdt_hash 00000040:00080000:2.0:1789762565.248051:0:8605:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 7 count 7 00000001:02000400:2.0:1789762565.891937:0:8762:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg348-server.virtnet: executing check_logdir /tmp/testlogs/ 00000100:00100000:2.0:1789762567.287305:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022105216 00000100:00100000:2.0:1789762567.287313:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022105216 00000100:00100000:2.0:1789762567.287376:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 9 00000100:00100000:2.0:1789762567.287388:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e90598700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022105216:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762567.287420:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762567.287475:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e90598700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022105216:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (206us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762567.287485:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 9 00000100:00100000:2.0:1789762567.778405:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014740608 00000100:00100000:2.0:1789762567.778412:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014740608 00000100:00100000:2.0:1789762567.778481:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 5 00000100:00100000:2.0:1789762567.778494:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014740608:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762567.778540:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014740608:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 50us (206us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762567.778547:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 5 00000100:00080000:2.0:1789762567.778579:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762567.778584:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762567.778588:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762567.799369:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022105344 00000100:00100000:2.0:1789762567.799375:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022105344 00000100:00100000:2.0:1789762567.799428:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10 00000100:00100000:2.0:1789762567.799445:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022105344:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762567.799477:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762567.799537:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022105344:12345-0@lo:13:osp-pre-1-0.0 Request processed in 100us (219us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762567.799545:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10 00000100:00100000:2.0:1789762567.799586:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 15 00000100:00100000:2.0:1789762567.799599:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9059b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022105600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762567.799640:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9059b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022105600:12345-0@lo:400:kworker.0 Request processed in 45us (316us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762567.799654:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 15 00000100:00080000:2.0:1789762567.799681:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762567.799687:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00080000:2.0:1789762567.799715:0:3271:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff981e851bb480 x1876702022105344/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1789762583 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762567.799752:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022105344:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762567.799796:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e9059bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022105984:0@lo:400:kworker.0 00010000:00010000:2.0:1789762568.119279:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789762570.126536:0:9394:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: OSS: 2.17.57.106 00010000:00010000:2.0:1789762572.215207:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762572.919218:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022106368:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762572.919289:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762572.919350:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e9059aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022107008:0@lo:400:kworker.0 00000100:00100000:2.0:1789762572.919414:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762572.919465:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022106368 00000100:00100000:2.0:1789762572.919471:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022106368 00000100:00100000:2.0:1789762572.919546:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12 00000100:00100000:2.0:1789762572.919561:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022106368:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762572.919597:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762572.919670:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022106368:12345-0@lo:13:osp-pre-1-0.0 Request processed in 107us (378us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762572.919694:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12 00000100:00080000:2.0:1789762572.919721:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00080000:2.0:1789762572.919769:0:3271:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff981e851bb480 x1876702022106368/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1789762588 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762572.919814:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851bb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022106368:0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1789762572.919912:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762572.920016:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762572.920136:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e9059aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022107008:0@lo:400:kworker.0 00000100:00080000:2.0:1789762572.920220:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762572.920370:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e9059b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022106752:0@lo:400:kworker.0 00010000:00010000:2.0:1789762576.311389:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762577.472847:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762577.479774:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762577.487450:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762577.487670:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014743040 00000100:00100000:2.0:1789762577.487675:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014743040 00000100:00100000:2.0:1789762577.487723:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 22 00000100:00100000:2.0:1789762577.487734:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014743040:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762577.487800:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014743040:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 64us (174us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762577.487819:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 22 00000100:00080000:2.0:1789762577.487849:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762577.487855:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762577.487871:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014743424 00000100:00100000:2.0:1789762577.487875:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014743424 00000100:00100000:2.0:1789762577.487916:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 14 00000100:00100000:2.0:1789762577.487936:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014743424:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762577.487975:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014743424:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 48us (183us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762577.487984:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 14 00000100:00100000:2.0:1789762577.488517:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014743168 00000100:00100000:2.0:1789762577.488522:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014743168 00000100:00100000:2.0:1789762577.488561:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 11 00000100:00100000:2.0:1789762577.488569:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+6:0:x1876702014743168:12345-192.168.203.48@tcp:41:mount.lustre.0 00000004:00000024:2.0:1789762577.488591:0:8165:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762577.488651:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+6:0:x1876702014743168:12345-192.168.203.48@tcp:41:mount.lustre.0 Request processed in 82us (179us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762577.488667:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 11 00000100:00100000:2.0:1789762577.490956:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014743936 00000100:00100000:2.0:1789762577.490963:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014743936 00000100:00100000:2.0:1789762577.490996:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 11 00000100:00100000:2.0:1789762577.491004:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0+-99:2362:x1876702014743936:12345-192.168.203.48@tcp:8:mount.lustre.0 00010000:00080000:2.0:1789762577.491043:0:6648:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from 871a7a42-3659-47e5-8fc9-f4f1e262590d@192.168.203.48@tcp t0 exp 0000000000000000 cur 100 last 0 00000020:00000080:2.0:1789762577.491073:0:6648:0:(genops.c:1385:class_connect()) connect: client 871a7a42-3659-47e5-8fc9-f4f1e262590d, cookie 0x416193e012b010d9 00002000:00100000:2.0:1789762577.491082:0:6648:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli 871a7a42-3659-47e5-8fc9-f4f1e262590d/ffff981ebc293800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 134217728 ocd_index: 0 ocd_group 0 00002000:00100000:2.0:1789762577.491128:0:6648:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:2.0:1789762577.491413:0:6648:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:2.0:1789762577.491435:0:6648:0:(import.c:65:import_set_state_nolock()) ffff981ebc292800 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789762577.491473:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+6:2362:x1876702014743936:12345-192.168.203.48@tcp:8:mount.lustre.0 Request processed in 466us (567us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762577.491483:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 11 00000100:00100000:2.0:1789762577.527275:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022107264:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762577.527332:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1789762577.527439:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00080000:2.0:1789762577.527580:0:3273:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff981ebfdc8e00 x1876702022107264/t0(0) o13->lustre-OST0000-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1789762593 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762577.527600:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdc8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022107264:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762578.039231:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762578.039253:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e90598000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022107776:0@lo:400:kworker.0 00000100:00100000:2.0:1789762578.039319:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762578.039388:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022107392:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762578.039433:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762578.039492:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022107776 00000100:00100000:2.0:1789762578.039498:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022107776 00000100:00100000:2.0:1789762578.039550:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022108032 00000100:00100000:2.0:1789762578.039554:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022108032 00000100:00100000:2.0:1789762578.039586:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022107648 00000100:00100000:2.0:1789762578.039591:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022107648 00000100:00100000:2.0:1789762578.039619:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23 00000100:00100000:2.0:1789762578.039631:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022107776:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762578.039680:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022107776:12345-0@lo:400:kworker.0 Request processed in 52us (361us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762578.039689:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23 00000100:00100000:2.0:1789762578.039715:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 25 00000100:00100000:2.0:1789762578.039724:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022107648:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762578.039785:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022107648:12345-0@lo:400:kworker.0 Request processed in 60us (316us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762578.039808:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 25 00000100:00100000:2.0:1789762578.039861:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e90598000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022107776:0@lo:400:kworker.0 00010000:00080000:2.0:1789762578.039887:0:3273:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff981e851b8000 x1876702022107392/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1789762594 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762578.039907:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022107392:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762578.040011:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e90599180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022107520:0@lo:400:kworker.0 00000001:00100000:2.0:1789762580.614248:0:6674:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 871a7a42-3659-47e5-8fc9-f4f1e262590d 00000100:00100000:2.0:1789762582.626691:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014744192 00000100:00100000:2.0:1789762582.626698:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014744192 00000100:00100000:2.0:1789762582.626815:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 199 00000100:00100000:2.0:1789762582.626832:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014744192:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762582.626920:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014744192:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 92us (316us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762582.626932:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 199 00000100:00100000:2.0:1789762582.629899:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014744448 00000100:00100000:2.0:1789762582.629908:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014744448 00000100:00100000:2.0:1789762582.630025:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 12 00000100:00100000:2.0:1789762582.630051:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2362:x1876702014744576:12345-192.168.203.48@tcp:9:kworker.0 00000020:00000080:2.0:1789762582.630084:0:6647:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x416193e012b00ff2 00000100:00080000:2.0:1789762582.630144:0:6647:0:(import.c:65:import_set_state_nolock()) ffff981e905d0000 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1789762582.630204:0:6647:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967298 00000100:00100000:2.0:1789762582.630709:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+2:2362:x1876702014744576:12345-192.168.203.48@tcp:9:kworker.0 Request processed in 670us (848us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762582.630717:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 12 00000100:00100000:2.0:1789762582.647216:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022108288:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762582.647268:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00080000:2.0:1789762582.647471:0:3273:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff981e851b8000 x1876702022108288/t0(0) o13->lustre-OST0000-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1789762598 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762582.647487:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022108288:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762583.159256:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762583.159287:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022108416:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762583.159437:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762583.159603:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022108416 00000100:00100000:2.0:1789762583.159613:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022108416 00000100:00100000:2.0:1789762583.159753:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 16 00000100:00100000:2.0:1789762583.159773:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022108416:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762583.159846:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762583.159967:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022108416:12345-0@lo:13:osp-pre-1-0.0 Request processed in 198us (537us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762583.160003:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 16 00000100:00100000:2.0:1789762583.160064:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 20 00000100:00100000:2.0:1789762583.160082:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022109184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762583.160223:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022109184:12345-0@lo:400:kworker.0 Request processed in 145us (540us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762583.160256:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 20 00010000:00080000:2.0:1789762583.160337:0:3273:0:(ldlm_request.c:1671:ldlm_cli_update_pool()) @@@ Zero SLV or limit found (SLV=0, limit=61549) req@ffff981e851b8000 x1876702022108416/t0(0) o13->lustre-OST0001-osc-MDT0000@0@lo:7/4 lens 224/368 e 0 to 0 dl 1789762599 ref 1 fl Rpc:RQU/200/0 rc 0/0 job:'osp-pre-1-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789762583.160366:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022108416:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762583.160474:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4ab7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022109056:0@lo:400:kworker.0 00000100:00100000:2.0:1789762583.160500:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4971180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022108544:0@lo:400:kworker.0 00000001:02000400:2.0:1789762584.526774:0:11163:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: Using TIMEOUT=20 00000100:00100000:2.0:1789762585.693033:0:5789:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022109440 00000100:00100000:2.0:1789762585.693048:0:5789:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022109440 00000100:00100000:2.0:1789762585.693192:0:5789:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21 00000100:00100000:2.0:1789762585.693227:0:5789:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+3:11310:x1876702022109440:12345-0@lo:104: 00010000:00010000:2.0:1789762585.693254:0:5789:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.148@tcp lock: ffff981e8ec04e00/0x416193e012b010a8 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b010af expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.693410:0:5789:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1789762585.693448:0:5789:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+4:11310:x1876702022109440:12345-0@lo:104: Request processed in 224us (495us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762585.693485:0:5789:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21 00000100:00100000:2.0:1789762585.693530:0:5789:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022109696 00000100:00100000:2.0:1789762585.693547:0:5789:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022109696 00000100:00100000:2.0:1789762585.693628:0:5789:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23 00000100:00100000:2.0:1789762585.693653:0:5789:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+4:11310:x1876702022109696:12345-0@lo:104: 00010000:00010000:2.0:1789762585.693677:0:5789:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.148@tcp lock: ffff981e80031e00/0x416193e012b0108c lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x416193e012b01093 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.693769:0:5789:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:2.0:1789762585.693809:0:5789:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+3:11310:x1876702022109696:12345-0@lo:104: Request processed in 154us (382us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762585.693863:0:5789:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23 00000100:00100000:2.0:1789762585.693961:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc00000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022109952:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789762585.694141:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022110080 00000100:00100000:2.0:1789762585.694150:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022110080 00010000:00010000:2.0:1789762585.694197:0:5790:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff981e880aba00/0x416193e012b010a1 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x416193e012b0109a expref: 19 pid: 5798 timeout: 258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762585.694282:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24 00000100:00100000:2.0:1789762585.694310:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022110080:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789762585.694334:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762585.694349:0:5790:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff981e880aba00/0x416193e012b010a1 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 8 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x416193e012b0109a expref: 19 pid: 5798 timeout: 258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.694372:0:5790:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff981e880aba00/0x416193e012b010a1 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 8 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x416193e012b0109a expref: 19 pid: 5798 timeout: 258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.694445:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff981e880aba00/0x416193e012b010a1 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 9 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x416193e012b0109a expref: 19 pid: 5798 timeout: 258 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.694493:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762585.694594:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+18:3274:x1876702022110080:12345-0@lo:103:ldlm_bl.0 Request processed in 281us (534us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762585.694629:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24 00000100:00100000:2.0:1789762585.694702:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022110208 00000100:00100000:2.0:1789762585.694715:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022110208 00010000:00010000:2.0:1789762585.694755:0:5790:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff981e87dfd400/0x416193e012b01093 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x416193e012b0108c expref: 18 pid: 5799 timeout: 258 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762585.694906:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc01500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022110080:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789762585.695496:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 28 00000100:00100000:2.0:1789762585.695518:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+12:2364:x1876702014745216:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762585.695538:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762585.695548:0:5790:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff981e880aa400/0x416193e012b01085 lrc: 4/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce83 expref: 12 pid: 5798 timeout: 208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695567:0:5790:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff981e880aa400/0x416193e012b01085 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce83 expref: 12 pid: 5798 timeout: 208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695587:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff981e880aa400/0x416193e012b01085 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bce83 expref: 12 pid: 5798 timeout: 208 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695606:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: MGS lock: ffff981e80030400/0x416193e012b010e0 lrc: 3/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 11310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695643:0:5790:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff981e80030400/0x416193e012b010e0 lrc: 4/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 11310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695670:0:5790:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff981e80030400/0x416193e012b010e0 lrc: 5/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695684:0:5790:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff981e80030400/0x416193e012b010e0 lrc: 5/0,1 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695698:0:5790:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGS lock: ffff981e80030400/0x416193e012b010e0 lrc: 4/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695718:0:5790:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGS lock: ffff981e80030400/0x416193e012b010e0 lrc: 5/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695735:0:5790:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762585.695743:0:5790:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: MGS lock: ffff981e80030400/0x416193e012b010e0 lrc: 5/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695758:0:5790:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: MGS lock: ffff981e80030400/0x416193e012b010e0 lrc: 6/0,0 mode: EX/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695805:0:5790:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGS lock: ffff981e80030400/0x416193e012b010e0 lrc: 4/0,0 mode: --/EX res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11310 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762585.695828:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762585.695897:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+11:2364:x1876702014745216:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 382us (546us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762585.695913:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 28 00000001:00100000:2.0:1789762585.736194:0:6674:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:2.0:1789762585.736215:0:6674:0:(genops.c:885:class_export_put()) final put ffff981ebc385800/f1bd28de-ee78-4d18-bad3-fa7cec968893 00000100:00100000:2.0:1789762587.107330:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762587.111114:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000020:01000000:2.0:1789762587.321153:0:5802:0:(obd_config.c:2145:class_config_parse_llog()) Processed log params gen 7-6 (rc=0) 10000000:01000000:2.0:1789762587.321179:0:5802:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.148@tcp: configuration from log 'params' succeeded (0). 00010000:00010000:2.0:1789762587.321185:0:5802:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.148@tcp lock: ffff981ec21d2200/0x416193e012b010f5 lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b010fc expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762587.321193:0:5802:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.148@tcp lock: ffff981ec21d2200/0x416193e012b010f5 lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b010fc expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789762587.321200:0:5802:0:(mgc_request.c:611:do_requeue()) updating log params 10000000:01000000:2.0:1789762587.321201:0:5802:0:(mgc_request.c:1802:mgc_process_log()) Process log params-ffff981e90361800 from 7 10000000:01000000:2.0:1789762587.321203:0:5802:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for params (res 0x736d61726170) 00010000:00010000:2.0:1789762587.321216:0:5802:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.148@tcp lock: ffff981e8ec05800/0x416193e012b01103 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762587.321222:0:5802:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.148@tcp lock: ffff981e8ec05800/0x416193e012b01103 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762587.321227:0:5802:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.148@tcp lock: ffff981e8ec05800/0x416193e012b01103 lrc: 3/1,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762587.321237:0:5802:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc868700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022110720:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1789762587.321263:0:5802:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762587.321285:0:5802:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e87b78cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762587.321486:0:5802:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc868700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022110720:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789762587.321497:0:5802:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.148@tcp lock: ffff981e8ec05800/0x416193e012b01103 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x416193e012b0110a expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762587.321513:0:5802:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.148@tcp lock: ffff981e8ec05800/0x416193e012b01103 lrc: 4/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x416193e012b0110a expref: -99 pid: 5802 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762587.321546:0:5802:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc868700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022110848:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789762587.321573:0:5802:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762587.321592:0:5802:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e87b78cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762587.322061:0:5802:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc868700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022110848:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789762587.322083:0:5802:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc868700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022110976:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789762587.322132:0:5802:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762587.322157:0:5802:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e87b78cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762587.322399:0:5802:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc868700 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:bbe4e1bd-8437-4448-83b9-0b3f7551a439:5802:1876702022110976:0@lo:503:ll_cfg_requeue.0 00000040:00080000:2.0:1789762587.323107:0:11526:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x2:0x0] index 64768 count 7 00000100:00100000:2.0:1789762587.746313:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762587.746346:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762587.746408:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762587.746462:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762587.746499:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762587.746564:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 32 00000100:00100000:2.0:1789762587.746584:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014746368:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762587.746642:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014746368:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 62us (179us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762587.746666:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 32 00000100:00100000:2.0:1789762588.099874:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014746752 00000100:00100000:2.0:1789762588.099892:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014746752 00000100:00100000:2.0:1789762588.099943:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 2 00000100:00100000:2.0:1789762588.099966:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:7630:x1876702014746752:12345-192.168.203.48@tcp:37:bash.0 00000100:00100000:2.0:1789762588.171159:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014747904 00000100:00100000:2.0:1789762588.171165:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014747904 00000100:00100000:2.0:1789762588.171195:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 19 00000100:00100000:2.0:1789762588.171202:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:10007:x1876702014747904:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:2.0:1789762588.171224:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762588.171235:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b01173 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf08 expref: 7 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.171246:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179800) 00010000:00010000:2.0:1789762588.171252:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762588.171264:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b0117a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.171270:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b0117a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.171276:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b0117a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762588.171407:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007ad14b3a reply state 000000008f983b35 transno 4294967298 ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b0117a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762588.171413:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762588.171417:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000100:00100000:2.0:1789762588.171459:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:10007:x1876702014747904:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 258us (396us total) trans 4294967298 rc 0/0 00000100:00100000:2.0:1789762588.171472:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 19 00000100:00100000:2.0:1789762588.181001:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014748416 00000100:00100000:2.0:1789762588.181007:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014748416 00000100:00100000:2.0:1789762588.181039:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 22 00000100:00100000:2.0:1789762588.181046:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:10008:x1876702014748416:12345-192.168.203.48@tcp:36:chown.0 00010000:00010000:2.0:1789762588.181067:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762588.181077:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b011a4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf16 expref: 9 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.181106:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1:0x0].0x0 (ffff981ec2179100) 00010000:00010000:2.0:1789762588.181112:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762588.181123:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b011ab lrc: 3/0,1 mode: --/PW res: [0x200000401:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.181130:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b011ab lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.181137:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b011ab lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.189592:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b011ab lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.189605:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b011ab lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.189614:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b011ab lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.189621:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec04a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762588.189633:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b011ab lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.189641:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b011ab lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.189654:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762588.189658:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762588.189662:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b011ab lrc: 2/0,0 mode: --/PW res: [0x200000401:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.189669:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b011ab lrc: 0/0,0 mode: --/PW res: [0x200000401:0x1:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.204230:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b011dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1:0x0].0xbcdd3602 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.204244:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b011dc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1:0x0].0xbcdd3602 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.204267:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b011dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1:0x0].0xbcdd3602 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.204272:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762588.204275:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b011dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1:0x0].0xbcdd3602 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.204281:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b011dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x1:0x0].0xbcdd3602 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.204292:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1:0x0].0xbcdd3602 (ffff981ec2179800) 00010000:00010000:2.0:1789762588.204295:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762588.204308:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b011dc lrc: 2/0,0 mode: --/PW res: [0x200000401:0x1:0x0].0xbcdd3602 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.204330:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b011dc lrc: 0/0,0 mode: --/PW res: [0x200000401:0x1:0x0].0xbcdd3602 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.205470:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b011f1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.205482:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b011f1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.205491:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b011f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.205499:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762588.205503:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b011f1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.205512:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b011f1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.205523:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:2.0:1789762588.205527:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762588.205531:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b011f1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.205539:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b011f1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762588.206793:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014749312 00000100:00100000:2.0:1789762588.206801:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014749312 00000100:00100000:2.0:1789762588.206842:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 14 00000100:00100000:2.0:1789762588.206861:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:0+-99:2362:x1876702014749312:12345-192.168.203.48@tcp:8:touch.500 00010000:00080000:2.0:1789762588.206889:0:6647:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0000: connection from f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp t0 exp 0000000000000000 cur 111 last 0 00000020:00000080:2.0:1789762588.206934:0:6647:0:(genops.c:1385:class_connect()) connect: client f1bd28de-ee78-4d18-bad3-fa7cec968893, cookie 0x416193e012b011ff 00002000:00100000:2.0:1789762588.206959:0:6647:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0000: cli f1bd28de-ee78-4d18-bad3-fa7cec968893/ffff981ebc295800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 3407872 ocd_index: 0 ocd_group 0 00002000:00100000:2.0:1789762588.206970:0:6647:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0000: cli supports cksum type f7, return f7 00002000:00080000:2.0:1789762588.207207:0:6647:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00000100:00080000:2.0:1789762588.207245:0:6647:0:(import.c:65:import_set_state_nolock()) ffff981ebc290000 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789762588.207298:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2362:x1876702014749312:12345-192.168.203.48@tcp:8:touch.500 Request processed in 438us (558us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762588.207309:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 14 00000100:00100000:2.0:1789762588.208144:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014749440 00000100:00100000:2.0:1789762588.208150:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014749440 00000100:00100000:2.0:1789762588.208190:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 15 00000100:00100000:2.0:1789762588.208198:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:0:x1876702014749440:12345-192.168.203.48@tcp:2:touch.500 00000100:00100000:2.0:1789762588.216695:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:0:x1876702014749440:12345-192.168.203.48@tcp:2:touch.500 Request processed in 8495us (8627us total) trans 4294967299 rc 0/0 00000100:00100000:2.0:1789762588.216702:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 15 00000100:00100000:2.0:1789762588.217594:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762588.225363:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762588.229939:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762588.231239:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762588.233526:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762588.234455:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762588.234465:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b01268 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.234506:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762588.234513:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b01268 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.234523:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b01268 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.234544:0:8466:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2:0x0].0x0 (ffff981ec2179600) 00010000:00010000:2.0:1789762588.234551:0:8466:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789762588.234554:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b01268 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.234563:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b01268 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.234577:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762588.234581:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762588.234584:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762588.235737:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762588.236662:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762588.237976:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65000/0x416193e012b01276 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.237984:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65000/0x416193e012b01276 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.237988:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65000/0x416193e012b01276 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.237991:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa65000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762588.237999:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65000/0x416193e012b01276 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.238004:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65000/0x416193e012b01276 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.238011:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x792b1d9 (ffff981ec2179f00) 00010000:00010000:2.0:1789762588.238013:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762588.238015:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65000/0x416193e012b01276 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762588.238020:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65000/0x416193e012b01276 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x792b1d9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762588.279227:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4ab6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022111872:0@lo:400:kworker.0 00000100:00100000:2.0:1789762588.279290:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762588.279400:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022111744 00000100:00100000:2.0:1789762588.279408:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022111744 00000100:00100000:2.0:1789762588.279480:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 18 00000100:00100000:2.0:1789762588.279495:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9059aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022111744:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762588.279534:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762588.279591:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9059aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022111744:12345-0@lo:13:osp-pre-1-0.0 Request processed in 102us (284us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762588.279599:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 18 00000100:00080000:2.0:1789762588.279630:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762588.279639:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762588.279664:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 34 00000100:00100000:2.0:1789762588.279678:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022112000:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762588.279748:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022112000:12345-0@lo:400:kworker.0 Request processed in 73us (430us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762588.279785:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 34 00000100:00100000:2.0:1789762588.279866:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4ab6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022111872:0@lo:400:kworker.0 00010000:00010000:2.0:1789762589.624195:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762589.649837:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762589.663611:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762589.682472:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014753920 00000100:00100000:2.0:1789762589.682481:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014753920 00000100:00100000:2.0:1789762589.682531:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 43 00000100:00100000:2.0:1789762589.682541:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:10411:x1876702014753920:12345-192.168.203.48@tcp:101:touch.0 00010000:00010000:2.0:1789762589.682557:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762589.682570:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b0131e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bcf78 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682632:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b01325 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682643:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b01325 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682650:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b01325 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682661:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b0132c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682667:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b0132c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682672:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b0132c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682745:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b01333 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682751:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b01333 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682756:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b01333 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762589.682773:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3:0x0]: returning size 16384 00000004:00200000:2.0:1789762589.682775:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762589.682800:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b01333 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682807:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b01325 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682812:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b01325 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682816:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b01325 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682821:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec04e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762589.682823:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b01325 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682828:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b01325 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682836:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179d00) 00010000:00010000:2.0:1789762589.682844:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762589.682848:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b01325 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682853:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b01325 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682862:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b0132c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682866:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b0132c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682871:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b0132c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682875:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec05400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762589.682878:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b0132c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682882:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b0132c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682896:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981e82a26e00) 00010000:00010000:2.0:1789762589.682898:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762589.682900:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b0132c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682904:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b0132c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682927:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b0131e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf78 expref: 22 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682963:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b01333 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf78 expref: 21 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.682969:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762589.682971:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789762589.682973:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05200, rc 0) 00000100:00100000:2.0:1789762589.683011:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:10411:x1876702014753920:12345-192.168.203.48@tcp:101:touch.0 Request processed in 473us (601us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762589.683016:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 43 00000100:00100000:2.0:1789762589.684111:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014754176 00000100:00100000:2.0:1789762589.684118:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014754176 00000100:00100000:2.0:1789762589.684159:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 44 00000100:00100000:2.0:1789762589.684169:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:10411:x1876702014754176:12345-192.168.203.48@tcp:101:touch.0 00010000:00010000:2.0:1789762589.684178:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762589.684187:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b0133a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bcf7f expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.684255:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b01341 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.684264:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b01341 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.684271:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b01341 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.684293:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01348 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.684304:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01348 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.684311:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01348 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762589.684784:0:8165:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x3:0x0], index 0: 0 00000004:00080000:2.0:1789762589.684793:0:8165:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x2:0x0], index 1: 0 00010000:00010000:2.0:1789762589.684952:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0134f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.684960:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0134f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.684967:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0134f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.684992:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b01341 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.684997:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b01341 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685002:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b01341 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685006:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec04800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762589.685008:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b01341 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685012:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b01341 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685026:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762589.685028:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762589.685031:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b01341 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685036:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b01341 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762589.685046:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cad36bd6 reply state 0000000050f268d7 transno 4294967314 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01348 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762589.685052:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762589.685071:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b0133a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf7f expref: 24 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685102:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0134f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf7f expref: 23 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685108:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff981e82a26000) 00010000:00010000:2.0:1789762589.685111:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762589.685113:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e1600, rc 0) 00000100:00100000:2.0:1789762589.685153:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:10411:x1876702014754176:12345-192.168.203.48@tcp:101:touch.0 Request processed in 987us (1116us total) trans 4294967314 rc 0/0 00000100:00100000:2.0:1789762589.685160:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 44 00010000:00010000:2.0:1789762589.685653:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01348 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685660:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01348 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685675:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01348 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685679:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762589.685682:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01348 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685686:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01348 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685693:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x77289514 (ffff981e82a26600) 00010000:00010000:2.0:1789762589.685695:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762589.685697:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01348 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.685701:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01348 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762589.686221:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762589.687134:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762589.688192:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014754688 00000100:00100000:2.0:1789762589.688199:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014754688 00000100:00100000:2.0:1789762589.688243:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 16 00000100:00100000:2.0:1789762589.688257:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014754688:12345-192.168.203.48@tcp:2:touch.0 00000100:00100000:2.0:1789762589.688501:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014754688:12345-192.168.203.48@tcp:2:touch.0 Request processed in 246us (347us total) trans 4294967300 rc 0/0 00000100:00100000:2.0:1789762589.688508:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 16 00000100:00100000:2.0:1789762589.699072:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014755200 00000100:00100000:2.0:1789762589.699101:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014755200 00000100:00100000:2.0:1789762589.699145:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 8 00000100:00100000:2.0:1789762589.699152:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:10411:x1876702014755200:12345-192.168.203.48@tcp:35:touch.0 00000100:00100000:2.0:1789762589.699381:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:10411:x1876702014755200:12345-192.168.203.48@tcp:35:touch.0 Request processed in 227us (353us total) trans 4294967316 rc 0/0 00000100:00100000:2.0:1789762589.699391:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 8 00000100:00100000:2.0:1789762589.719355:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014755456 00000100:00100000:2.0:1789762589.719366:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014755456 00000100:00100000:2.0:1789762589.719414:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 47 00000100:00100000:2.0:1789762589.719422:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:10415:x1876702014755456:12345-192.168.203.48@tcp:101:touch.0 00010000:00010000:2.0:1789762589.719442:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762589.719457:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01372 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bcf8d expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.719548:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01379 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.719561:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01379 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.719570:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01379 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.719612:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01380 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3:0x0].0x992ad74f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.719617:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01380 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.719622:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01380 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762589.720000:0:8165:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x3:0x0], index 1: 0 00000004:00080000:2.0:1789762589.720005:0:8165:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x4:0x0], index 0: 0 00010000:00010000:2.0:1789762589.720149:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0000/0x416193e012b01387 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720156:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0000/0x416193e012b01387 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720161:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0000/0x416193e012b01387 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720182:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01379 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720188:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01379 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720192:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01379 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720196:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762589.720199:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01379 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720204:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01379 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720212:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762589.720214:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762589.720217:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01379 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720227:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01379 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762589.720236:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c3f74e9e reply state 0000000016ca280d transno 4294967317 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01380 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762589.720243:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762589.720263:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01372 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf8d expref: 28 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720273:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0000/0x416193e012b01387 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf8d expref: 27 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720278:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffff981e82a26e00) 00010000:00010000:2.0:1789762589.720279:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762589.720281:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e0000, rc 0) 00000100:00100000:2.0:1789762589.720331:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:10415:x1876702014755456:12345-192.168.203.48@tcp:101:touch.0 Request processed in 910us (1082us total) trans 4294967317 rc 0/0 00000100:00100000:2.0:1789762589.720336:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 47 00010000:00010000:2.0:1789762589.720986:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01380 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720994:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01380 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.720999:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01380 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.721003:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e1400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762589.721011:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01380 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.721016:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01380 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.721023:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x992ad74f (ffff981e82a26600) 00010000:00010000:2.0:1789762589.721025:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762589.721027:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01380 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.721031:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01380 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762589.724157:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014756096 00000100:00100000:2.0:1789762589.724164:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014756096 00000100:00100000:2.0:1789762589.724202:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 20 00000100:00100000:2.0:1789762589.724211:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014756096:12345-192.168.203.48@tcp:2:touch.0 00000100:00100000:2.0:1789762589.724403:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2363:x1876702014756096:12345-192.168.203.48@tcp:2:touch.0 Request processed in 193us (279us total) trans 4294967301 rc 0/0 00000100:00100000:2.0:1789762589.724412:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 20 00000100:00100000:2.0:1789762589.725299:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762589.739744:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014756608 00000100:00100000:2.0:1789762589.739753:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014756608 00000100:00100000:2.0:1789762589.739792:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 50 00000100:00100000:2.0:1789762589.739801:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:10419:x1876702014756608:12345-192.168.203.48@tcp:101:bash.0 00010000:00010000:2.0:1789762589.739815:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762589.739825:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b013a3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bcf9b expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.739894:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b013aa lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.739903:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b013aa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.739910:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b013aa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762589.739917:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3:0x0]: returning size 16384 00010000:00010000:2.0:1789762589.740010:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b013a3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf9b expref: 32 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.740027:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b013aa lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf9b expref: 31 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.740037:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762589.740040:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1789762589.740043:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e0c00, rc 0) 00000100:00100000:2.0:1789762589.740083:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:10419:x1876702014756608:12345-192.168.203.48@tcp:101:bash.0 Request processed in 284us (428us total) trans 4294967320 rc 0/0 00000100:00100000:2.0:1789762589.740119:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 50 00000100:00100000:2.0:1789762589.742705:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014756864 00000100:00100000:2.0:1789762589.742711:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014756864 00000100:00100000:2.0:1789762589.742746:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 10 00000100:00100000:2.0:1789762589.742754:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b1bc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:10419:x1876702014756864:12345-192.168.203.48@tcp:37:bash.0 00000100:00100000:2.0:1789762589.744574:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b1bc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:10419:x1876702014756864:12345-192.168.203.48@tcp:37:bash.0 Request processed in 1820us (1925us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762589.744585:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 10 00000100:00100000:2.0:1789762589.752480:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014757248 00000100:00100000:2.0:1789762589.752486:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014757248 00000100:00100000:2.0:1789762589.752517:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 22 00000100:00100000:2.0:1789762589.752526:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014757248:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:2.0:1789762589.752538:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762589.752550:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a66c43a3/0x416193e012b013e9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bcfb7 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762589.752566:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x2:0x0] initial LVB size: 0, mtime: 0x6aad9c1d, atime: 0x6aad9c1d, ctime: 0x6aad9c1d, blocks: 0x1 00010000:00010000:2.0:1789762589.752580:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=113s v=(0 0) 00010000:00010000:2.0:1789762589.752584:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762589.752593:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a66c43a3/0x416193e012b013e9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfb7 expref: 7 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.752606:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a66c43a3/0x416193e012b013e9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfb7 expref: 7 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.752614:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e1400, rc 0) 00000100:00100000:2.0:1789762589.752641:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2365:x1876702014757248:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 116us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762589.752648:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 22 00000100:00100000:2.0:1789762589.753733:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014757504 00000100:00100000:2.0:1789762589.753739:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014757504 00000100:00100000:2.0:1789762589.753767:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 53 00000100:00100000:2.0:1789762589.753774:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:10419:x1876702014757504:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:2.0:1789762589.753801:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762589.753812:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65e00/0x416193e012b013db lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfa9 expref: 33 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.753822:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff981ea0aa3a00) 00010000:00010000:2.0:1789762589.753825:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762589.753837:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b013f0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.753843:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b013f0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.753849:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b013f0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762589.753974:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000009f42141 reply state 00000000e613d383 transno 4294967321 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b013f0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762589.753981:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762589.754033:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:10419:x1876702014757504:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 258us (355us total) trans 4294967321 rc 0/0 00000100:00100000:2.0:1789762589.754042:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 53 00010000:00010000:2.0:1789762589.754613:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b013f0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.754622:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b013f0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.754627:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b013f0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.754639:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762589.754642:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b013f0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.754646:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b013f0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.754653:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff981ea0aa3a00) 00010000:00010000:2.0:1789762589.754655:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762589.754658:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b013f0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.754662:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b013f0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762589.754987:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014757632 00000100:00100000:2.0:1789762589.754994:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014757632 00000100:00100000:2.0:1789762589.755040:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 24 00000100:00100000:2.0:1789762589.755051:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014757632:12345-192.168.203.48@tcp:2:chmod.0 00002000:00010000:2.0:1789762589.755192:0:6647:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x3:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1789762589.755196:0:6647:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x3:0x0] updating lvb mtime from disk: 1789762589 -> 1789762589 00002000:00010000:2.0:1789762589.755198:0:6647:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x3:0x0] updating lvb atime from disk: 1789762589 -> 1789762589 00002000:00010000:2.0:1789762589.755201:0:6647:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x3:0x0] updating lvb ctime from disk: 1789762589 -> 1789762589 00002000:00010000:2.0:1789762589.755203:0:6647:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x3:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1789762589.755236:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2364:x1876702014757632:12345-192.168.203.48@tcp:2:chmod.0 Request processed in 188us (305us total) trans 4294967302 rc 0/0 00000100:00100000:2.0:1789762589.755243:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 24 00000100:00100000:2.0:1789762589.757472:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014758144 00000100:00100000:2.0:1789762589.757484:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014758144 00000100:00100000:2.0:1789762589.757513:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 25 00000100:00100000:2.0:1789762589.757522:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2364:x1876702014758144:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:2.0:1789762589.757530:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762589.757540:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000007c3976/0x416193e012b01413 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bcfcc expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762589.757553:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x4:0x0] initial LVB size: 0, mtime: 0x6aad9c1d, atime: 0x6aad9c1d, ctime: 0x6aad9c1d, blocks: 0x1 00010000:00010000:2.0:1789762589.757565:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=113s v=(0 0) 00010000:00010000:2.0:1789762589.757571:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762589.757579:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000007c3976/0x416193e012b01413 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfcc expref: 10 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.757592:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000007c3976/0x416193e012b01413 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfcc expref: 10 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.757601:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e0200, rc 0) 00000100:00100000:2.0:1789762589.757633:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014758144:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 112us (206us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762589.757645:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 25 00000100:00100000:2.0:1789762589.775865:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014758912 00000100:00100000:2.0:1789762589.775872:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014758912 00000100:00100000:2.0:1789762589.775908:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 56 00000100:00100000:2.0:1789762589.775916:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:10423:x1876702014758912:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:2.0:1789762589.775930:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762589.775944:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1789762589.775969:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b013bf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfa2 expref: 34 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.775983:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762589.775987:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762589.775997:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01428 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bcfd3 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776116:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1800/0x416193e012b0142f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776126:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1800/0x416193e012b0142f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776133:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1800/0x416193e012b0142f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776142:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1200/0x416193e012b01436 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776147:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1200/0x416193e012b01436 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776152:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1200/0x416193e012b01436 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776191:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0143d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776197:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0143d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776210:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0143d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762589.776216:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x5:0x0]: returning size 0 00010000:00010000:2.0:1789762589.776286:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1800/0x416193e012b0142f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776296:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1800/0x416193e012b0142f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776303:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1800/0x416193e012b0142f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776310:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e1800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762589.776314:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1800/0x416193e012b0142f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776319:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1800/0x416193e012b0142f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776326:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762589.776328:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762589.776330:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1800/0x416193e012b0142f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776334:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1800/0x416193e012b0142f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776341:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1200/0x416193e012b01436 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776345:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1200/0x416193e012b01436 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776354:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1200/0x416193e012b01436 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776358:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e1200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762589.776360:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1200/0x416193e012b01436 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776364:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1200/0x416193e012b01436 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776369:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x77289514 (ffff981e82a26500) 00010000:00010000:2.0:1789762589.776370:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762589.776372:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1200/0x416193e012b01436 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776375:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1200/0x416193e012b01436 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776413:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01428 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfd3 expref: 36 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776422:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0143d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfd3 expref: 35 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.776427:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff981ea0aa3a00) 00010000:00010000:2.0:1789762589.776428:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762589.776430:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e0a00, rc 0) 00000100:00100000:2.0:1789762589.776463:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:10423:x1876702014758912:12345-192.168.203.48@tcp:101:dd.500 Request processed in 549us (648us total) trans 4294967323 rc 0/0 00000100:00100000:2.0:1789762589.776477:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 56 00010000:00010000:2.0:1789762589.778331:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b0144b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.778339:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b0144b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.778344:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b0144b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.778348:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aac00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762589.778351:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b0144b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.778355:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b0144b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.778362:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff981ea0aa3a00) 00010000:00010000:2.0:1789762589.778364:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762589.778367:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b0144b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.778371:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b0144b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762589.780735:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014759552 00000100:00100000:2.0:1789762589.780741:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014759552 00000100:00100000:2.0:1789762589.780780:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 30 00000100:00100000:2.0:1789762589.780787:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:10423:x1876702014759552:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:2.0:1789762589.780794:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762589.780808:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000896c7f19/0x416193e012b01459 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bcfe1 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.780824:0:6649:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a66c43a3/0x416193e012b013e9 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfb7 expref: 11 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.780831:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762589.780833:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762589.780836:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000896c7f19/0x416193e012b01459 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000080000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfe1 expref: 11 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.780851:0:6649:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a66c43a3/0x416193e012b013e9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfb7 expref: 11 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.780859:0:6649:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a66c43a3/0x416193e012b013e9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfb7 expref: 11 pid: 6649 timeout: 213 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762589.780867:0:6649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4bcd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6649:1876702022112640:192.168.203.48@tcp:104: 00000100:00100000:2.0:1789762589.780902:0:6649:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e87ad4e40 going to sleep for 11 seconds 00000100:00100000:2.0:1789762589.781400:0:6649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4bcd500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6649:1876702022112640:192.168.203.48@tcp:104: 00010000:00010000:2.0:1789762589.781412:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000896c7f19/0x416193e012b01459 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfe1 expref: 11 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.781421:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e1000, rc 0) 00000100:00100000:2.0:1789762589.781446:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:10423:x1876702014759552:12345-192.168.203.48@tcp:101:dd.500 Request processed in 661us (761us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762589.781459:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 30 00000100:00100000:2.0:1789762589.781558:0:8405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014759680 00000100:00100000:2.0:1789762589.781562:0:8405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014759680 00010000:00010000:2.0:1789762589.781576:0:8405:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a66c43a3/0x416193e012b013e9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfb7 expref: 11 pid: 6649 timeout: 213 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762589.781600:0:8405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 30 00000100:00100000:2.0:1789762589.781606:0:8405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:2364:x1876702014759680:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762589.781612:0:8405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762589.781616:0:8405:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a66c43a3/0x416193e012b013e9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfb7 expref: 11 pid: 6649 timeout: 213 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.781625:0:8405:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a66c43a3/0x416193e012b013e9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfb7 expref: 11 pid: 6649 timeout: 213 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.781636:0:8405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a66c43a3/0x416193e012b013e9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfb7 expref: 11 pid: 6649 timeout: 213 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.781643:0:8405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762589.781646:0:8405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762589.781649:0:8405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000896c7f19/0x416193e012b01459 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfe1 expref: 10 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.781655:0:8405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000896c7f19/0x416193e012b01459 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfe1 expref: 10 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762589.781687:0:8405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000896c7f19/0x416193e012b01459 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfe1 expref: 10 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762589.781699:0:8405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4bcd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:8405:1876702022112768:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762589.781736:0:8405:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e87ad4e40 going to sleep for 11 seconds 00000100:00100000:2.0:1789762589.782369:0:8405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4bcd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:8405:1876702022112768:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762589.782378:0:8405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762589.782401:0:8405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014759680:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 796us (879us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762589.782407:0:8405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 30 00000100:00100000:2.0:1789762589.782594:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762589.792416:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762589.798766:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:2365:x1876702014760064:12345-192.168.203.48@tcp:4:dd.500 Request processed in 6233us (6351us total) trans 4294967305 rc 0/0 00000100:00100000:2.0:1789762589.798777:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 2 00000100:00100000:2.0:1789762589.798782:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 113286984916 time 113096432315 state 0x2 ref 0 00002000:00100000:2.0:1789762589.798790:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x3:0x0] (0->1048575) req@ffff981eb4bcd180 x1876702014760064/t4294967305(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:370/0 lens 488/448 e 0 to 0 dl 1789762600 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762589.798798:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcd180 with x1876702014760064 ext(0->1048575) 00002000:00010000:2.0:1789762589.798801:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981eb4bcd180 00000100:00100000:2.0:1789762589.809324:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762589.831485:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762590.192706:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 113696609383 time 113496609383 state 0x1 ref 4 00000100:00100000:2.0:1789762590.192747:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:2366:x1876702014760320:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762590.196001:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:2366:x1876702014760320:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3278us (386676us total) trans 4294967307 rc 0/0 00000100:00100000:2.0:1789762590.196009:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 4 00002000:00100000:2.0:1789762590.196019:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x3:0x0] (2097152->3145727) req@ffff981eb4bcdf80 x1876702014760320/t4294967307(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:370/0 lens 488/448 e 0 to 0 dl 1789762600 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762590.196027:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcdf80 with x1876702014760320 ext(2097152->3145727) 00002000:00010000:2.0:1789762590.196033:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981eb4bcdf80 00000100:00100000:2.0:1789762590.615010:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762590.637375:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762590.798886:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014761600 00000100:00100000:2.0:1789762590.798894:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014761600 00002000:00100000:2.0:1789762590.798922:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x2:0x0] (4194304->5242879) req@ffff981e902dca80 x1876702014761600/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:371/0 lens 488/0 e 0 to 0 dl 1789762601 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762590.798943:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e902dca80 with x1876702014761600 ext(4194304->5242879) 00002000:00010000:2.0:1789762590.798947:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e902dca80 00000100:00100000:2.0:1789762590.798957:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 114296916842 time 114096916842 state 0x1 ref 7 00000100:00100000:2.0:1789762590.806454:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762590.993210:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 114497107103 time 114297107103 state 0x1 ref 8 00000100:00100000:2.0:1789762590.993260:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:2366:x1876702014760832:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762590.996399:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:2366:x1876702014760832:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3151us (395635us total) trans 4294967306 rc 0/0 00000100:00100000:2.0:1789762590.996416:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 8 00002000:00100000:2.0:1789762590.996425:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x2:0x0] (2097152->3145727) req@ffff981ebac26300 x1876702014760832/t4294967306(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:371/0 lens 488/448 e 0 to 0 dl 1789762601 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762590.996433:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac26300 with x1876702014760832 ext(2097152->3145727) 00002000:00010000:2.0:1789762590.996439:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac26300 00000100:00080000:2.0:1789762591.008610:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762591.008617:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762591.048745:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014762752 00000100:00100000:2.0:1789762591.048755:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014762752 00000100:00100000:2.0:1789762591.051380:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762591.051590:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762591.051607:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033d7c9c2/0x416193e012b01491 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcffd expref: 25 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.051648:0:6649:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b01483 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 5 type: EXT [1048576->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfef expref: 25 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.051663:0:6649:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b01483 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 5 type: EXT [1048576->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfef expref: 25 pid: 6648 timeout: 214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762591.051676:0:6649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead6e4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6649:1876702022113280:192.168.203.48@tcp:104: 00000100:00100000:0.0:1789762591.051719:0:6649:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e9967f9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762591.052708:0:6649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead6e4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6649:1876702022113280:192.168.203.48@tcp:104: 00010000:00010000:2.0:1789762591.052729:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033d7c9c2/0x416193e012b01491 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcffd expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.052742:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ec21d3200, rc 0) 00000100:00100000:2.0:1789762591.052783:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:10436:x1876702014763008:12345-192.168.203.48@tcp:101:dd.0 Request processed in 1262us (1407us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762591.052801:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 33 00000100:00100000:2.0:1789762591.057623:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762591.061643:0:5792:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014763392 00000100:00100000:2.0:1789762591.061652:0:5792:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014763392 00010000:00010000:2.0:1789762591.061679:0:5792:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000007c3976/0x416193e012b01413 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfcc expref: 25 pid: 6649 timeout: 214 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762591.061713:0:5792:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 32 00000100:00100000:2.0:1789762591.061732:0:5792:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86f04850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2365:x1876702014763392:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762591.061744:0:5792:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762591.061759:0:5792:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000007c3976/0x416193e012b01413 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfcc expref: 25 pid: 6649 timeout: 214 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.061770:0:5792:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000007c3976/0x416193e012b01413 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfcc expref: 25 pid: 6649 timeout: 214 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.061787:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000007c3976/0x416193e012b01413 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x4:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfcc expref: 25 pid: 6649 timeout: 214 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.061799:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762591.061803:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762591.061808:0:5792:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0149f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd00b expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.061817:0:5792:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0149f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd00b expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.061841:0:5792:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0149f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd00b expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762591.061851:0:5792:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc01180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022113536:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762591.061917:0:5792:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e87ad4e40 going to sleep for 11 seconds 00000100:00100000:0.0:1789762591.076817:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762591.076906:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014763520 00000100:00100000:2.0:1789762591.076919:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014763520 00000100:00100000:2.0:1789762591.076985:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 36 00000100:00100000:2.0:1789762591.076992:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:10443:x1876702014763520:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762591.077001:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762591.077011:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000007c3976/0x416193e012b014a6 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd012 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.077029:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x3:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762591.077032:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762591.077036:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000007c3976/0x416193e012b014a6 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 6 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd012 expref: 26 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.077045:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000007c3976/0x416193e012b014a6 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 6 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd012 expref: 26 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.077053:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e0200, rc 0) 00000100:00100000:2.0:1789762591.077082:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:10443:x1876702014763520:12345-192.168.203.48@tcp:101:dd.0 Request processed in 91us (267us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762591.077108:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 36 00000100:00100000:2.0:1789762591.077604:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762591.080597:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762591.089009:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762591.089060:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014763904 00000100:00100000:2.0:1789762591.089067:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014763904 00000100:00100000:2.0:1789762591.089119:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 39 00000100:00100000:2.0:1789762591.089128:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:10446:x1876702014763904:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762591.089138:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762591.089146:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008cb18182/0x416193e012b014bb lrc: 2/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd027 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.089174:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x3:0x0].0x0 since=0s v=(5 0) 00010000:00010000:2.0:1789762591.089177:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:2.0:1789762591.089182:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008cb18182/0x416193e012b014bb lrc: 3/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd027 expref: 29 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.089190:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008cb18182/0x416193e012b014bb lrc: 3/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd027 expref: 29 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762591.089197:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e1a00, rc 0) 00000100:00100000:2.0:1789762591.089228:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:10446:x1876702014763904:12345-192.168.203.48@tcp:101:dd.0 Request processed in 102us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762591.089234:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 39 00000100:00100000:2.0:1789762591.109152:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762591.132362:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762591.132442:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014764160 00000100:00100000:0.0:1789762591.132451:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014764160 00002000:00100000:0.0:1789762591.132486:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (1048576->2097151) req@ffff981ebfc03800 x1876702014764160/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/0 e 0 to 0 dl 1789762602 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.132494:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfc03800 with x1876702014764160 ext(1048576->2097151) 00002000:00010000:0.0:1789762591.132498:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfc03800 00000100:00100000:0.0:1789762591.132507:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114413339138 time 114413401848 state 0x0 ref 1 00000100:00100000:0.0:1789762591.132534:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114413501848 time 114413401848 state 0x1 ref 1 00000100:00100000:0.0:1789762591.132553:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114413501848 time 114436463985 state 0x0 ref 1 00000100:00100000:0.0:1789762591.132563:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:2365:x1876702014764160:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762591.135911:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762591.143075:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762591.143130:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762591.146533:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2363:x1876702014764416:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3283us (3457us total) trans 4294967312 rc 0/0 00000100:00100000:0.0:1789762591.146542:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 21 00002000:00100000:0.0:1789762591.146552:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x4:0x0] (6291456->7340031) req@ffff981ead6e7480 x1876702014764416/t4294967312(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/448 e 0 to 0 dl 1789762602 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.146560:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ead6e7480 with x1876702014764416 ext(6291456->7340031) 00002000:00010000:0.0:1789762591.146565:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ead6e7480 00000100:00100000:2.0:1789762591.151124:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762591.151702:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2363:x1876702014764672:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3391us (3514us total) trans 4294967314 rc 0/0 00000100:00100000:2.0:1789762591.151710:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 23 00002000:00100000:2.0:1789762591.151718:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (3145728->4194303) req@ffff981e902de300 x1876702014764672/t4294967314(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/448 e 0 to 0 dl 1789762602 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762591.151726:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e902de300 with x1876702014764672 ext(3145728->4194303) 00002000:00010000:2.0:1789762591.151730:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e902de300 00000100:00100000:2.0:1789762591.153748:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014764928 00000100:00100000:2.0:1789762591.153757:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014764928 00002000:00100000:2.0:1789762591.153795:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x4:0x0] (7340032->8388607) req@ffff981e90599880 x1876702014764928/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/0 e 0 to 0 dl 1789762602 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762591.153805:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e90599880 with x1876702014764928 ext(7340032->8388607) 00002000:00010000:2.0:1789762591.153809:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e90599880 00000100:00100000:2.0:1789762591.153845:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114455314007 time 114455214007 state 0x1 ref 2 00000100:00100000:2.0:1789762591.153878:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114455314007 time 114457788430 state 0x0 ref 2 00000100:00100000:2.0:1789762591.153900:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e90599880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2366:x1876702014764928:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762591.154318:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2364:x1876702014764800:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3004us (3194us total) trans 4294967315 rc 0/0 00000100:00100000:0.0:1789762591.154327:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 24 00002000:00100000:0.0:1789762591.154335:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x4:0x0] (4194304->5242879) req@ffff981ebfc00e00 x1876702014764800/t4294967315(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/448 e 0 to 0 dl 1789762602 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.154344:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfc00e00 with x1876702014764800 ext(4194304->5242879) 00002000:00010000:0.0:1789762591.154348:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfc00e00 00000100:00100000:2.0:1789762591.156814:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e90599880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2366:x1876702014764928:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2925us (3137us total) trans 4294967316 rc 0/0 00000100:00100000:2.0:1789762591.156820:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 25 00000100:00100000:2.0:1789762591.156825:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114455314007 time 114457788430 state 0x2 ref 0 00002000:00100000:2.0:1789762591.156834:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x4:0x0] (7340032->8388607) req@ffff981e90599880 x1876702014764928/t4294967316(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/448 e 0 to 0 dl 1789762602 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762591.156841:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e90599880 with x1876702014764928 ext(7340032->8388607) 00002000:00010000:2.0:1789762591.156845:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e90599880 00000100:00100000:0.0:1789762591.200066:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902df100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:2366:x1876702014760960:12345-192.168.203.48@tcp:4:dd.500 Request processed in 6716us (592096us total) trans 4294967307 rc 0/0 00000100:00100000:0.0:1789762591.200073:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 9 00002000:00100000:0.0:1789762591.200082:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x2:0x0] (3145728->4194303) req@ffff981e902df100 x1876702014760960/t4294967307(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:371/0 lens 488/448 e 0 to 0 dl 1789762601 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762591.200113:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e902df100 with x1876702014760960 ext(3145728->4194303) 00002000:00010000:0.0:1789762591.200118:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e902df100 00000100:00100000:0.0:1789762591.204582:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014765056 00000100:00100000:0.0:1789762591.204588:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014765056 00002000:00100000:0.0:1789762591.204612:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x3:0x0] (2097152->3145727) req@ffff981e902ddc00 x1876702014765056/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/0 e 0 to 0 dl 1789762602 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.204631:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e902ddc00 with x1876702014765056 ext(2097152->3145727) 00010000:00010000:0.0:1789762591.204639:0:6654:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b01483 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [1048576->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfef expref: 31 pid: 6648 timeout: 214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.204653:0:6654:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b01483 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [1048576->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfef expref: 31 pid: 6648 timeout: 214 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762591.204670:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981e902ddc00 00000100:00100000:0.0:1789762591.204684:0:6654:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff981ebf63ba00 key uid={0} rule@0xffff981e97832b00 gen 0 rate 10000 depth 3 token 3 deadline 0 time 114508593671 state 0x0 ref 1 00000100:00100000:0.0:1789762591.204696:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf63ba00 key uid={0} rule@0xffff981e97832b00 gen 0 rate 10000 depth 3 token 3 deadline 114508693671 time 114508593671 state 0x1 ref 1 00000100:00100000:2.0:1789762591.204710:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf63ba00 key uid={0} rule@0xffff981e97832b00 gen 0 rate 10000 depth 3 token 2 deadline 114508693671 time 114508618286 state 0x0 ref 1 00000100:00100000:2.0:1789762591.204727:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2364:x1876702014765056:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762591.212679:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2364:x1876702014765184:12345-192.168.203.48@tcp:4:dd.0 Request processed in 7980us (8182us total) trans 4294967308 rc 0/0 00000100:00100000:0.0:1789762591.212687:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 26 00000100:00100000:0.0:1789762591.212691:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114457888430 time 114508595750 state 0x2 ref 0 00002000:00100000:0.0:1789762591.212701:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x3:0x0] (0->1048575) req@ffff981e902dc700 x1876702014765184/t4294967308(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/448 e 0 to 0 dl 1789762602 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.212709:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e902dc700 with x1876702014765184 ext(0->1048575) 00002000:00010000:0.0:1789762591.212713:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e902dc700 00000100:00100000:0.0:1789762591.214573:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902ddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2364:x1876702014765056:12345-192.168.203.48@tcp:4:dd.0 Request processed in 9850us (10041us total) trans 4294967309 rc 0/0 00000100:00100000:0.0:1789762591.214588:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 0 00000100:00100000:0.0:1789762591.214592:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf63ba00 key uid={0} rule@0xffff981e97832b00 gen 0 rate 10000 depth 3 token 2 deadline 114508693671 time 114508618286 state 0x2 ref 0 00002000:00100000:0.0:1789762591.214600:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x3:0x0] (2097152->3145727) req@ffff981e902ddc00 x1876702014765056/t4294967309(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/448 e 0 to 0 dl 1789762602 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.214607:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e902ddc00 with x1876702014765056 ext(2097152->3145727) 00010000:00010000:0.0:1789762591.214614:0:11723:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b01483 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [1048576->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfef expref: 34 pid: 6648 timeout: 214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.214626:0:11723:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b01483 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [1048576->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfef expref: 34 pid: 6648 timeout: 214 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762591.214632:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981e902ddc00 00000100:00100000:0.0:1789762591.216007:0:7595:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014765312 00000100:00100000:0.0:1789762591.216014:0:7595:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014765312 00010000:00010000:0.0:1789762591.216035:0:7595:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b01483 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [1048576->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfef expref: 34 pid: 6648 timeout: 214 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762591.216066:0:7595:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 33 00000100:00100000:0.0:1789762591.216073:0:7595:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2365:x1876702014765312:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762591.216083:0:7595:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1789762591.216120:0:7595:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x3:0x0] updating lvb size from disk: 0 -> 3145728 00002000:00010000:0.0:1789762591.216124:0:7595:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x3:0x0] updating lvb mtime from disk: 1789762589 -> 1789762591 00002000:00010000:0.0:1789762591.216126:0:7595:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x3:0x0] updating lvb ctime from disk: 1789762589 -> 1789762591 00010000:00010000:0.0:1789762591.216131:0:7595:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b01483 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 10 type: EXT [1048576->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfef expref: 34 pid: 6648 timeout: 214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216150:0:7595:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b01483 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 10 type: EXT [1048576->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfef expref: 34 pid: 6648 timeout: 214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216171:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b01483 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x3:0x0].0x0 rrc: 10 type: EXT [1048576->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfef expref: 34 pid: 6648 timeout: 214 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216185:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216188:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216196:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000033d7c9c2/0x416193e012b01491 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcffd expref: 33 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216204:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033d7c9c2/0x416193e012b01491 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcffd expref: 33 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216211:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216213:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216217:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000020801b7b/0x416193e012b01498 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd004 expref: 33 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216224:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000020801b7b/0x416193e012b01498 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd004 expref: 33 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216228:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216230:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216234:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000007c3976/0x416193e012b014a6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd012 expref: 33 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216245:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000007c3976/0x416193e012b014a6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd012 expref: 33 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216251:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216252:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216256:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000087ea9cad/0x416193e012b014ad lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd019 expref: 33 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216261:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000087ea9cad/0x416193e012b014ad lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd019 expref: 33 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216266:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216268:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216271:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b014b4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd020 expref: 33 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216277:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b014b4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd020 expref: 33 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216282:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216283:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x3:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762591.216287:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008cb18182/0x416193e012b014bb lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd027 expref: 33 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216292:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008cb18182/0x416193e012b014bb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd027 expref: 33 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762591.216324:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008cb18182/0x416193e012b014bb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd027 expref: 33 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762591.216336:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead6e5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022113664:192.168.203.48@tcp:105: 00010000:00010000:0.0:1789762591.216401:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b014b4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd020 expref: 33 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762591.216412:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead6e5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022113792:192.168.203.48@tcp:105: 00010000:00010000:0.0:1789762591.216451:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000087ea9cad/0x416193e012b014ad lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd019 expref: 33 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762591.216466:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead6e5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022113920:192.168.203.48@tcp:105: 00010000:00010000:0.0:1789762591.216501:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000007c3976/0x416193e012b014a6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd012 expref: 33 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762591.216512:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead6e4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022114048:192.168.203.48@tcp:105: 00010000:00010000:0.0:1789762591.216559:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000020801b7b/0x416193e012b01498 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd004 expref: 33 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762591.216566:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead6e7b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022114176:192.168.203.48@tcp:105: 00010000:00010000:0.0:1789762591.216588:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000033d7c9c2/0x416193e012b01491 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcffd expref: 33 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762591.216621:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead6e6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022114304:192.168.203.48@tcp:105: 00000100:00100000:0.0:1789762591.216643:0:7595:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e9967f9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789762591.218155:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead6e5c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022113664:192.168.203.48@tcp:105: 00000100:00100000:0.0:1789762591.218168:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead6e5f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022113792:192.168.203.48@tcp:105: 00000100:00100000:0.0:1789762591.218175:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead6e5880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022113920:192.168.203.48@tcp:105: 00000100:00100000:0.0:1789762591.218180:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead6e4380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022114048:192.168.203.48@tcp:105: 00000100:00100000:0.0:1789762591.218185:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead6e7b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022114176:192.168.203.48@tcp:105: 00000100:00100000:0.0:1789762591.218701:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead6e6d80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022114304:192.168.203.48@tcp:105: 00010000:00010000:0.0:1789762591.218726:0:7595:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762591.218762:0:7595:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2365:x1876702014765312:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 2688us (2818us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762591.218768:0:7595:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 33 00000100:00100000:0.0:1789762591.254334:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014765440 00000100:00100000:0.0:1789762591.254341:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014765440 00002000:00100000:0.0:1789762591.254368:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x3:0x0] (3145728->4194303) req@ffff981e902dd880 x1876702014765440/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/0 e 0 to 0 dl 1789762602 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.254376:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e902dd880 with x1876702014765440 ext(3145728->4194303) 00002000:00010000:0.0:1789762591.254380:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e902dd880 00000100:00100000:0.0:1789762591.254388:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114457888430 time 114508595750 state 0x0 ref 1 00000100:00100000:0.0:1789762591.254419:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114508695750 time 114508595750 state 0x1 ref 1 00000100:00100000:2.0:1789762591.254464:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114508695750 time 114558369982 state 0x0 ref 1 00000100:00100000:2.0:1789762591.254483:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2366:x1876702014765440:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762591.256939:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2366:x1876702014765440:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2444us (2670us total) trans 4294967310 rc 0/0 00000100:00100000:0.0:1789762591.256947:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 27 00000100:00100000:0.0:1789762591.256951:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114508695750 time 114558369982 state 0x2 ref 0 00002000:00100000:0.0:1789762591.256960:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x3:0x0] (3145728->4194303) req@ffff981e902dd880 x1876702014765440/t4294967310(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/448 e 0 to 0 dl 1789762602 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.256967:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e902dd880 with x1876702014765440 ext(3145728->4194303) 00002000:00010000:0.0:1789762591.256970:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e902dd880 00000100:00100000:0.0:1789762591.258295:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014765568 00000100:00100000:0.0:1789762591.258301:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014765568 00002000:00100000:0.0:1789762591.258319:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x3:0x0] (4194304->5242879) req@ffff981ebac24e00 x1876702014765568/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/0 e 0 to 0 dl 1789762602 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.258327:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac24e00 with x1876702014765568 ext(4194304->5242879) 00002000:00010000:0.0:1789762591.258330:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac24e00 00000100:00100000:0.0:1789762591.258335:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114508695750 time 114558369982 state 0x0 ref 1 00000100:00100000:0.0:1789762591.258358:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114558469982 time 114558369982 state 0x1 ref 1 00000100:00100000:0.0:1789762591.258373:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114558469982 time 114562284782 state 0x0 ref 1 00000100:00100000:0.0:1789762591.258384:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2366:x1876702014765568:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762591.260616:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2366:x1876702014765568:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2233us (2399us total) trans 4294967311 rc 0/0 00000100:00100000:2.0:1789762591.260629:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 28 00000100:00100000:2.0:1789762591.260633:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114558469982 time 114562284782 state 0x2 ref 0 00002000:00100000:2.0:1789762591.260642:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x3:0x0] (4194304->5242879) req@ffff981ebac24e00 x1876702014765568/t4294967311(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/448 e 0 to 0 dl 1789762602 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762591.260649:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac24e00 with x1876702014765568 ext(4194304->5242879) 00002000:00010000:2.0:1789762591.260652:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac24e00 00000100:00100000:0.0:1789762591.271177:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014765696 00000100:00100000:0.0:1789762591.271184:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014765696 00002000:00100000:0.0:1789762591.271204:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x3:0x0] (5242880->6291455) req@ffff981ebac25500 x1876702014765696/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/0 e 0 to 0 dl 1789762602 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.271222:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac25500 with x1876702014765696 ext(5242880->6291455) 00002000:00010000:0.0:1789762591.271225:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac25500 00000100:00100000:0.0:1789762591.271231:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114558469982 time 114562284782 state 0x0 ref 1 00000100:00100000:0.0:1789762591.271255:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114562384782 time 114562284782 state 0x1 ref 1 00000100:00100000:0.0:1789762591.271279:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114562384782 time 114575189714 state 0x0 ref 1 00000100:00100000:0.0:1789762591.271290:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2363:x1876702014765696:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762591.275183:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014765824 00000100:00100000:2.0:1789762591.275189:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014765824 00002000:00100000:2.0:1789762591.275215:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x3:0x0] (7340032->8388607) req@ffff981ead4f4700 x1876702014765824/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/0 e 0 to 0 dl 1789762602 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762591.275227:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ead4f4700 with x1876702014765824 ext(7340032->8388607) 00002000:00010000:2.0:1789762591.275233:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ead4f4700 00000100:00100000:2.0:1789762591.275276:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114575289714 time 114575189714 state 0x1 ref 2 00000100:00100000:2.0:1789762591.275293:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114575289714 time 114579204121 state 0x0 ref 2 00000100:00100000:2.0:1789762591.275306:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2366:x1876702014765824:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762591.275519:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762591.275562:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014766208 00000100:00100000:2.0:1789762591.275567:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014766208 00000100:00100000:2.0:1789762591.275600:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 12 00000100:00100000:2.0:1789762591.275612:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:10436:x1876702014766208:12345-192.168.203.48@tcp:35:dd.0 00000001:00080000:2.0:1789762591.275782:0:5808:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967329 00000100:00100000:2.0:1789762591.275856:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:10436:x1876702014766208:12345-192.168.203.48@tcp:35:dd.0 Request processed in 248us (340us total) trans 4294967330 rc 0/0 00000100:00100000:2.0:1789762591.275870:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 12 00000020:00080000:2.0:1789762591.275874:0:5808:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789762591.276777:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762591.276797:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762591.276815:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014765952 00000100:00100000:0.0:1789762591.276821:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014765952 00002000:00100000:0.0:1789762591.276842:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x3:0x0] (1048576->2097151) req@ffff981ebfc03480 x1876702014765952/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/0 e 0 to 0 dl 1789762602 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.276851:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfc03480 with x1876702014765952 ext(1048576->2097151) 00002000:00010000:0.0:1789762591.276871:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfc03480 00000100:00100000:0.0:1789762591.276903:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114579304121 time 114579204121 state 0x1 ref 3 00000100:00100000:0.0:1789762591.276937:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114580933660 time 114580833660 state 0x1 ref 3 00000100:00100000:0.0:1789762591.276955:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2364:x1876702014765952:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762591.277740:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:2366:x1876702014765824:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2435us (2602us total) trans 4294967313 rc 0/0 00000100:00100000:0.0:1789762591.277768:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 30 00002000:00100000:0.0:1789762591.277787:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x3:0x0] (7340032->8388607) req@ffff981ead4f4700 x1876702014765824/t4294967313(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:372/0 lens 488/448 e 0 to 0 dl 1789762602 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762591.277818:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ead4f4700 with x1876702014765824 ext(7340032->8388607) 00002000:00010000:0.0:1789762591.277826:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ead4f4700 00000100:00100000:0.0:1789762591.277868:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf462000 key uid={0} rule@0xffff981eb4ce9a00 gen 0 rate 10000 depth 3 token 2 deadline 114580933660 time 114581778573 state 0x0 ref 2 00000100:00100000:0.0:1789762591.277876:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:2364:x1876702014766336:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762591.593701:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 115097599203 time 114897599203 state 0x1 ref 7 00000100:00100000:2.0:1789762591.593756:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2366:x1876702014761216:12345-192.168.203.48@tcp:4:dd.500 00010000:00010000:2.0:1789762591.671287:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762591.998139:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:2365:x1876702014761472:12345-192.168.203.48@tcp:4:dd.500 Request processed in 4005us (1360760us total) trans 4294967320 rc 0/0 00000100:00100000:2.0:1789762591.998154:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 13 00002000:00100000:2.0:1789762591.998170:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x3:0x0] (7340032->8388607) req@ffff981eb4bcf100 x1876702014761472/t4294967320(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:371/0 lens 488/448 e 0 to 0 dl 1789762601 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762591.998184:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcf100 with x1876702014761472 ext(7340032->8388607) 00002000:00010000:2.0:1789762591.998194:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981eb4bcf100 00000100:00100000:2.0:1789762592.198271:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+46:2364:x1876702014761600:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3935us (1399422us total) trans 4294967316 rc 0/0 00000100:00100000:2.0:1789762592.198292:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 14 00002000:00100000:2.0:1789762592.198314:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x2:0x0] (4194304->5242879) req@ffff981e902dca80 x1876702014761600/t4294967316(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:371/0 lens 488/448 e 0 to 0 dl 1789762601 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762592.198337:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e902dca80 with x1876702014761600 ext(4194304->5242879) 00002000:00010000:2.0:1789762592.198349:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e902dca80 00000100:00100000:2.0:1789762592.794625:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 116098449210 time 116098527242 state 0x0 ref 1 00000100:00100000:2.0:1789762592.794677:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+47:2364:x1876702014761984:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762592.833291:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762592.833379:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:0.0:1789762592.833483:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762592.833511:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 3 00000100:00100000:2.0:1789762592.833534:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:6050:x1876702014766592:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762592.833554:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00002000:00010000:2.0:1789762592.833573:0:5791:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x3:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:2.0:1789762592.833577:0:5791:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x3:0x0] updating lvb mtime from disk: 1789762589 -> 1789762590 00002000:00010000:2.0:1789762592.833579:0:5791:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x3:0x0] updating lvb ctime from disk: 1789762589 -> 1789762590 00010000:00010000:2.0:1789762592.833605:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e180c044/0x416193e012b01452 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801080020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcfda expref: 44 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762592.833665:0:5791:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:2.0:1789762592.833684:0:5791:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb mtime from disk: 1789762589 -> 1789762591 00002000:00010000:2.0:1789762592.833688:0:5791:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x4:0x0] updating lvb ctime from disk: 1789762589 -> 1789762591 00010000:00010000:2.0:1789762592.833701:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0149f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd00b expref: 43 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.833753:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762592.833818:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:6050:x1876702014766592:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 289us (433us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762592.833833:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 3 00000100:00100000:0.0:1789762592.863057:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762592.863134:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014766848 00000100:00100000:2.0:1789762592.863144:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014766848 00000100:00100000:2.0:1789762592.863191:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 59 00000100:00100000:2.0:1789762592.863203:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:10471:x1876702014766848:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762592.863243:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762592.863260:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0600/0x416193e012b014c2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd02e expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863316:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1e00/0x416193e012b014c9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863327:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1e00/0x416193e012b014c9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863336:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1e00/0x416193e012b014c9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863346:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0e00/0x416193e012b014d0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863351:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0e00/0x416193e012b014d0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863356:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0e00/0x416193e012b014d0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863489:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0400/0x416193e012b014d7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863498:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0400/0x416193e012b014d7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863507:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0400/0x416193e012b014d7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762592.863518:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x5:0x0]: returning size 16777216 00010000:00010000:2.0:1789762592.863609:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1e00/0x416193e012b014c9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863616:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1e00/0x416193e012b014c9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863620:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1e00/0x416193e012b014c9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863624:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e1e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762592.863628:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1e00/0x416193e012b014c9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863633:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1e00/0x416193e012b014c9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863642:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762592.863644:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762592.863647:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1e00/0x416193e012b014c9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863652:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1e00/0x416193e012b014c9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863660:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0e00/0x416193e012b014d0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863665:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0e00/0x416193e012b014d0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863669:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0e00/0x416193e012b014d0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863680:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762592.863682:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0e00/0x416193e012b014d0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863686:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0e00/0x416193e012b014d0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863691:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x77289514 (ffff981e82a26f00) 00010000:00010000:2.0:1789762592.863693:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762592.863694:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0e00/0x416193e012b014d0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863698:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0e00/0x416193e012b014d0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863727:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0600/0x416193e012b014c2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd02e expref: 45 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863737:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0400/0x416193e012b014d7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd02e expref: 44 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.863743:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff981ea0aa3a00) 00010000:00010000:2.0:1789762592.863745:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762592.863747:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e0400, rc 0) 00000100:00100000:2.0:1789762592.863792:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:10471:x1876702014766848:12345-192.168.203.48@tcp:101:dd.0 Request processed in 593us (744us total) trans 4294967331 rc 0/0 00000100:00100000:2.0:1789762592.863798:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 59 00000100:00100000:0.0:1789762592.865297:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762592.865339:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762592.865378:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762592.865436:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014767104 00000100:00100000:2.0:1789762592.865452:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014767104 00000100:00100000:0.0:1789762592.865477:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:0.0:1789762592.865500:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:10471:x1876702014767232:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762592.865516:0:10916:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762592.865535:0:10916:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b014f3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x3:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd03c expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762592.865641:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762592.865647:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762592.865656:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b014f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd03c expref: 47 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762592.865678:0:10916:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b014f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd03c expref: 47 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762592.865727:0:10916:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ec21d2400, rc 0) 00000100:00100000:0.0:1789762592.865766:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+47:10471:x1876702014767232:12345-192.168.203.48@tcp:101:dd.0 Request processed in 269us (420us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762592.865784:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:0.0:1789762592.866990:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762592.867015:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762592.867030:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014767360 00000100:00100000:2.0:1789762592.867036:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014767360 00000100:00100000:0.0:1789762592.867059:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762592.867082:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 218 00000100:00100000:2.0:1789762592.867119:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014767360:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762592.867128:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762592.867166:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014767744 00000100:00100000:0.0:1789762592.867172:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014767744 00000100:00100000:2.0:1789762592.867179:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014767360:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 58us (183us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762592.867192:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 218 00000100:00100000:0.0:1789762592.867233:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 25 00000100:00100000:0.0:1789762592.867248:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014767744:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:2.0:1789762592.867269:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762592.867279:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762592.867285:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762592.867307:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014767744:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 62us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762592.867325:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 25 00000100:00100000:2.0:1789762592.870304:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000004:00080000:2.0:1789762592.887265:0:6868:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 116191175337ns, next at 117191172699ns, rc = 0 00000100:00100000:0.0:1789762592.887268:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc01180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022114432:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762592.887341:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1789762592.887433:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762592.887599:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc01180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022114432:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762592.900080:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762592.909737:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762592.911703:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762592.911748:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014768384 00000100:00100000:2.0:1789762592.911753:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014768384 00000100:00100000:2.0:1789762592.911783:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 14 00000100:00100000:2.0:1789762592.911791:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+46:10487:x1876702014768384:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:0.0:1789762592.915259:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762592.915336:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014768512 00000100:00100000:2.0:1789762592.915342:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014768512 00000100:00100000:2.0:1789762592.915375:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 61 00000100:00100000:2.0:1789762592.915382:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+46:10487:x1876702014768512:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:2.0:1789762592.915420:0:8165:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7719616512 avail 7688159232 00000100:00100000:2.0:1789762592.915459:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+46:10487:x1876702014768512:12345-192.168.203.48@tcp:41:rm.0 Request processed in 78us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762592.915464:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 61 00000100:00100000:0.0:1789762592.916236:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762592.917464:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762592.917482:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b0151d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.917502:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aaa00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762592.917512:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b0151d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.917535:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b0151d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.917555:0:6663:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x6:0x0].0x0 (ffff981e87ddf300) 00010000:00010000:2.0:1789762592.917566:0:6663:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789762592.917571:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b0151d lrc: 1/0,0 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.917582:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b0151d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x6:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.917602:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762592.917605:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762592.917607:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762592.918623:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762592.919487:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762592.919506:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65400/0x416193e012b01539 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762592.919520:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa65400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762592.919527:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65400/0x416193e012b01539 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762592.919540:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65400/0x416193e012b01539 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762592.919554:0:5793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff981ec2179700) 00010000:00010000:0.0:1789762592.919558:0:5793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762592.919563:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65400/0x416193e012b01539 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762592.919569:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65400/0x416193e012b01539 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x5:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762592.919584:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762592.919588:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762592.919591:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762592.920100:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b0152b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.920109:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b0152b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.920115:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b0152b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.920120:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa65600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762592.920123:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b0152b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.920135:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b0152b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.920143:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x77289514 (ffff981ec2179e00) 00010000:00010000:2.0:1789762592.920146:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762592.920148:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b0152b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.920153:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b0152b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762592.921482:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762592.922475:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762592.922505:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014769536 00000100:00100000:2.0:1789762592.922511:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014769536 00000100:00100000:2.0:1789762592.922538:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 64 00000100:00100000:2.0:1789762592.922544:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:10487:x1876702014769536:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:2.0:1789762592.922567:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762592.922577:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b013aa lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf9b expref: 44 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.922590:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b01333 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf78 expref: 43 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.922597:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762592.922605:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65c00/0x416193e012b012d1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bcf63 expref: 42 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.922611:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179d00) 00010000:00010000:2.0:1789762592.922622:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762592.922633:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01540 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.922639:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01540 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.922645:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01540 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.922743:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01547 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.922756:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01547 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.922763:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01547 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.922814:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0154e lrc: 3/0,1 mode: --/EX res: [0x200000401:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.922837:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0154e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.922844:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0154e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762592.923214:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3:0x0]: returning size 16384 00010000:00010000:2.0:1789762592.923223:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0154e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923228:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0154e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923238:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0154e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923243:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762592.923245:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0154e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923250:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0154e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923258:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x0 (ffff981e82a26300) 00010000:00010000:2.0:1789762592.923260:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762592.923262:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0154e lrc: 2/0,0 mode: --/EX res: [0x200000401:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923267:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0154e lrc: 0/0,0 mode: --/EX res: [0x200000401:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923281:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01540 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923285:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01540 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923289:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01540 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923293:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762592.923295:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01540 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923306:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01540 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923312:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26f00) 00010000:00010000:2.0:1789762592.923313:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762592.923315:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01540 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762592.923319:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01540 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762592.923327:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000037971538 reply state 00000000e198dd5c transno 4294967336 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1400/0x416193e012b01547 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762592.923333:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762592.923377:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:10487:x1876702014769536:12345-192.168.203.48@tcp:36:rm.0 Request processed in 831us (903us total) trans 4294967336 rc 0/0 00000100:00100000:2.0:1789762592.923386:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 64 00000100:00100000:0.0:1789762593.194074:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762593.194194:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762593.195448:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762593.195486:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014769920 00000100:00100000:2.0:1789762593.195494:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014769920 00000100:00100000:2.0:1789762593.195536:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 66 00000100:00100000:2.0:1789762593.195547:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:10498:x1876702014769920:12345-192.168.203.48@tcp:36:mkdir.0 00010000:00010000:2.0:1789762593.195580:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762593.195598:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b0155c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd051 expref: 43 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.195613:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e87ddf700) 00010000:00010000:2.0:1789762593.195629:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762593.195693:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01563 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.195703:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01563 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.195710:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01563 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.195719:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0156a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.195724:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0156a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.195729:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0156a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762593.196288:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x9:0x0]: returning size 16384 00010000:00010000:2.0:1789762593.196295:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01563 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196300:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01563 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196305:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01563 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196310:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762593.196313:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01563 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196326:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01563 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196334:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26300) 00010000:00010000:2.0:1789762593.196336:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762593.196339:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01563 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196344:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01563 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762593.196360:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b826c90d reply state 000000007022e7d3 transno 4294967337 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0156a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762593.196368:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762593.196411:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:10498:x1876702014769920:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 867us (964us total) trans 4294967337 rc 0/0 00000100:00100000:2.0:1789762593.196417:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 66 00010000:00010000:2.0:1789762593.196880:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0156a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196889:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0156a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196894:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0156a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196898:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762593.196901:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0156a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196928:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0156a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196936:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981e82a26b00) 00010000:00010000:2.0:1789762593.196938:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762593.196941:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0156a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.196945:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b0156a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762593.201543:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014770048 00000100:00100000:2.0:1789762593.201548:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014770048 00000100:00100000:2.0:1789762593.201582:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 67 00000100:00100000:2.0:1789762593.201589:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9059bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:10499:x1876702014770048:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:2.0:1789762593.201599:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762593.201607:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b01571 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd058 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.201633:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01578 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.201640:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01578 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.201647:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01578 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762593.201660:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762593.201668:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762593.201681:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01578 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.201697:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0a00/0x416193e012b01571 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd058 expref: 45 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.201708:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0800/0x416193e012b01578 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd058 expref: 44 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.201714:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762593.201716:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762593.201718:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e0800, rc 0) 00000100:00100000:2.0:1789762593.201750:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9059bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:10499:x1876702014770048:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 162us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762593.201755:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 67 00000100:00100000:0.0:1789762593.207458:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014771200 00000100:00100000:0.0:1789762593.207466:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014771200 00000100:00100000:0.0:1789762593.207514:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 16 00000100:00100000:0.0:1789762593.207525:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b1a050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+46:10499:x1876702014771200:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:0.0:1789762593.207641:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b1a050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+47:10499:x1876702014771200:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 120us (217us total) trans 4294967340 rc 0/0 00000100:00100000:0.0:1789762593.207656:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 16 00000100:00100000:0.0:1789762593.210930:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762593.212937:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b015e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.212946:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b015e8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.212951:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b015e8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.212955:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa65600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762593.212958:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b015e8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.212963:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b015e8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.212973:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff981ec2179700) 00010000:00010000:0.0:1789762593.212975:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762593.212978:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b015e8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.212982:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65600/0x416193e012b015e8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x9:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762593.218427:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014771968 00000100:00100000:0.0:1789762593.218432:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014771968 00000100:00100000:0.0:1789762593.218465:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 75 00000100:00100000:0.0:1789762593.218476:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+49:10501:x1876702014771968:12345-192.168.203.48@tcp:101:touch.0 00010000:00010000:0.0:1789762593.218497:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762593.218510:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b0160b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd082 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.218571:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01612 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.218579:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01612 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.218585:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01612 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.218595:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01619 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x9:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.218599:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01619 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.218604:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01619 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762593.218973:0:6611:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x5:0x0], index 0: 0 00010000:00010000:0.0:1789762593.219136:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b01620 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.219144:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b01620 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.219150:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b01620 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.219179:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01612 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.219184:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01612 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.219189:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01612 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.219193:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762593.219196:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01612 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.219201:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01612 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.219212:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:0.0:1789762593.219214:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762593.219217:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01612 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.219221:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01612 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762593.219230:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009b01ba36 reply state 00000000513eee9d transno 4294967342 ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01619 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762593.219234:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762593.219253:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b0160b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd082 expref: 52 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.219274:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b01620 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd082 expref: 51 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.219279:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762593.219281:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762593.219283:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81483400, rc 0) 00000100:00100000:0.0:1789762593.219320:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+52:10501:x1876702014771968:12345-192.168.203.48@tcp:101:touch.0 Request processed in 849us (966us total) trans 4294967342 rc 0/0 00000100:00100000:0.0:1789762593.219326:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 75 00000100:00100000:0.0:1789762593.220738:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762593.223189:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762593.224216:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014772736 00000100:00100000:0.0:1789762593.224222:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014772736 00000100:00100000:0.0:1789762593.224445:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 17 00000100:00100000:0.0:1789762593.224455:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+52:10501:x1876702014772736:12345-192.168.203.48@tcp:35:touch.0 00000100:00100000:0.0:1789762593.224634:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+53:10501:x1876702014772736:12345-192.168.203.48@tcp:35:touch.0 Request processed in 180us (545us total) trans 4294967344 rc 0/0 00000100:00100000:0.0:1789762593.224641:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 17 00000100:00100000:0.0:1789762593.227806:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014772864 00000100:00100000:0.0:1789762593.227813:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014772864 00000100:00100000:0.0:1789762593.227851:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 78 00000100:00100000:0.0:1789762593.227860:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4970e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+53:10502:x1876702014772864:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:0.0:1789762593.227870:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762593.227880:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b0163c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd090 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.227938:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01643 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.227946:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01643 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.227953:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01643 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.227963:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0164a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.227968:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0164a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.227974:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0164a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228027:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b01651 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228033:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b01651 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228038:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b01651 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762593.228054:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb:0x0]: returning size 0 00000004:00200000:0.0:1789762593.228057:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762593.228068:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b01651 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228106:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b01651 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228112:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01643 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228117:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01643 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228122:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01643 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228127:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762593.228129:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01643 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228135:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01643 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228144:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:0.0:1789762593.228147:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762593.228149:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01643 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228154:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01643 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228163:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0164a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228167:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0164a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228177:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0164a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228181:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762593.228183:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0164a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228188:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0164a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228195:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x4009f790 (ffff981ea0aa3700) 00010000:00010000:0.0:1789762593.228197:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762593.228199:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0164a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228203:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0164a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228228:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b0163c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd090 expref: 55 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228238:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b01651 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd090 expref: 54 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.228243:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff981e87ddf500) 00010000:00010000:0.0:1789762593.228245:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762593.228247:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81482000, rc 0) 00000100:00100000:0.0:1789762593.228280:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4970e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+54:10502:x1876702014772864:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 422us (538us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762593.228294:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 78 00000100:00100000:0.0:1789762593.248960:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014773632 00000100:00100000:0.0:1789762593.248966:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014773632 00000100:00100000:0.0:1789762593.248998:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 80 00000100:00100000:0.0:1789762593.249004:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+54:10508:x1876702014773632:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762593.249015:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762593.249024:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b01666 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd09e expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249119:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0166d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249128:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0166d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249135:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0166d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249142:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01674 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249147:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01674 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249152:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01674 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249186:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b0167b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249201:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b0167b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249206:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b0167b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762593.249212:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb:0x0]: returning size 0 00010000:00010000:0.0:1789762593.249259:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0166d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249264:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0166d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249268:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0166d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249272:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762593.249275:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0166d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249280:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0166d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249286:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:0.0:1789762593.249288:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762593.249291:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0166d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249295:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0166d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249302:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01674 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249311:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01674 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249315:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01674 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249318:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762593.249320:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01674 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249324:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01674 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249329:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x4009f790 (ffff981ea0aa3700) 00010000:00010000:0.0:1789762593.249330:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762593.249332:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01674 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249335:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01674 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249360:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b01666 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd09e expref: 57 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249368:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b0167b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd09e expref: 56 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.249373:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff981e87ddf500) 00010000:00010000:0.0:1789762593.249375:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762593.249380:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81482e00, rc 0) 00000100:00100000:0.0:1789762593.249410:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+56:10508:x1876702014773632:12345-192.168.203.48@tcp:101:dd.500 Request processed in 406us (543us total) trans 4294967346 rc 0/0 00000100:00100000:0.0:1789762593.249416:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 80 00000100:00100000:0.0:1789762593.250356:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014773888 00000100:00100000:0.0:1789762593.250361:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014773888 00000100:00100000:0.0:1789762593.250388:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 81 00000100:00100000:0.0:1789762593.250394:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+56:10508:x1876702014773888:12345-192.168.203.48@tcp:36:dd.500 00010000:00010000:0.0:1789762593.250415:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762593.250426:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b0167b lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd09e expref: 56 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.250437:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff981e87ddf500) 00010000:00010000:0.0:1789762593.250439:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762593.250538:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01689 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.250547:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01689 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.250553:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01689 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762593.250586:0:6611:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967347 00000004:00010000:0.0:1789762593.250601:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009ace6dcd reply state 00000000b5fadf68 transno 4294967348 ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01689 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762593.250607:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762593.250649:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+58:10508:x1876702014773888:12345-192.168.203.48@tcp:36:dd.500 Request processed in 255us (325us total) trans 4294967348 rc 0/0 00000100:00100000:0.0:1789762593.250663:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 81 00000020:00080000:0.0:1789762593.250667:0:6611:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1789762593.251157:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01689 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.251164:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01689 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.251169:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01689 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.251173:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762593.251175:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01689 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.251179:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01689 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.251185:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff981e87ddf500) 00010000:00010000:0.0:1789762593.251187:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762593.251189:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01689 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.251193:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01689 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762593.252405:0:8405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014774144 00000100:00100000:2.0:1789762593.252410:0:8405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014774144 00010000:00010000:2.0:1789762593.252430:0:8405:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000692812a9/0x416193e012b01658 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd097 expref: 49 pid: 6648 timeout: 216 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762593.252455:0:8405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 34 00000100:00100000:2.0:1789762593.252461:0:8405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+49:2363:x1876702014774144:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762593.252470:0:8405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762593.252473:0:8405:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000692812a9/0x416193e012b01658 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd097 expref: 49 pid: 6648 timeout: 216 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.252487:0:8405:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000692812a9/0x416193e012b01658 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd097 expref: 49 pid: 6648 timeout: 216 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.252510:0:8405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000692812a9/0x416193e012b01658 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd097 expref: 49 pid: 6648 timeout: 216 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.252522:0:8405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762593.252526:0:8405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762593.252532:0:8405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000b7854f1/0x416193e012b01690 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0a5 expref: 48 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.252541:0:8405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b7854f1/0x416193e012b01690 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080080020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0a5 expref: 48 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.252562:0:8405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000b7854f1/0x416193e012b01690 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0a5 expref: 48 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762593.252570:0:8405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc01180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:8405:1876702022114688:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762593.252605:0:8405:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e9f6db9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762593.253316:0:8405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc01180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0000_UUID:8405:1876702022114688:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762593.253324:0:8405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762593.253346:0:8405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+48:2363:x1876702014774144:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 886us (1021us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762593.253352:0:8405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 34 00000100:00100000:0.0:1789762593.253491:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014774272 00000100:00100000:0.0:1789762593.253498:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014774272 00002000:00010000:0.0:1789762593.253520:0:6652:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/5 (0->18446744073709551615) 00002000:00100000:0.0:1789762593.253524:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbaa00 with x1876702014774272 ext(0->18446744073709551615) 00002000:00010000:0.0:1789762593.253530:0:6652:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdbaa00. 00000100:00100000:0.0:1789762593.253545:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 116098449210 time 116098527242 state 0x0 ref 1 00000100:00100000:0.0:1789762593.253554:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 116298527242 time 116098527242 state 0x1 ref 1 00000100:00100000:0.0:1789762593.253573:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 1 deadline 116298527242 time 116557483145 state 0x0 ref 1 00000100:00100000:0.0:1789762593.253581:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+48:2363:x1876702014774272:12345-192.168.203.48@tcp:10:dd.500 00002000:00010000:0.0:1789762593.253709:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1789762593.253712:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb mtime from disk: 1789762593 -> 1789762593 00002000:00010000:0.0:1789762593.253714:0:6652:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb atime from disk: 1789762593 -> 1789762593 00002000:00010000:0.0:1789762593.253716:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb ctime from disk: 1789762593 -> 1789762593 00002000:00010000:0.0:1789762593.253717:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:0.0:1789762593.253745:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+49:2363:x1876702014774272:12345-192.168.203.48@tcp:10:dd.500 Request processed in 165us (282us total) trans 4294967323 rc 0/0 00000100:00100000:0.0:1789762593.253752:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 34 00000100:00100000:0.0:1789762593.253756:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 1 deadline 116298527242 time 116557483145 state 0x2 ref 0 00002000:00010000:0.0:1789762593.253761:0:6652:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/5 (0->18446744073709551615) 00002000:00100000:0.0:1789762593.253763:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbaa00 with x1876702014774272 ext(0->18446744073709551615) 00002000:00010000:0.0:1789762593.253767:0:6652:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdbaa00. 00000100:00100000:0.0:1789762593.256204:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762593.256236:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014774400 00000100:00100000:2.0:1789762593.256242:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014774400 00000100:00100000:2.0:1789762593.256261:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 1 deadline 116298527242 time 116557483145 state 0x0 ref 1 00000100:00100000:2.0:1789762593.256274:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 1 deadline 116757483145 time 116557483145 state 0x1 ref 1 00000100:00100000:2.0:1789762593.256293:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 116757483145 time 116560204104 state 0x0 ref 1 00000100:00100000:2.0:1789762593.256301:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+49:2364:x1876702014774400:12345-192.168.203.48@tcp:4:dd.500 00010000:00010000:2.0:1789762593.256318:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01697 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.256328:0:6654:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000b7854f1/0x416193e012b01690 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0a5 expref: 49 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.256334:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762593.256337:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789762593.256339:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01697 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.256352:0:6654:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000b7854f1/0x416193e012b01690 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000080020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0a5 expref: 49 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.256360:0:6654:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000b7854f1/0x416193e012b01690 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000480020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0a5 expref: 49 pid: 10916 timeout: 216 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762593.256368:0:6654:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc01180 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0000_UUID:6654:1876702022114816:192.168.203.48@tcp:104: 00000100:00100000:2.0:1789762593.256401:0:6654:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e9f6db9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762593.257019:0:6654:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc01180 pname:cluuid:pid:xid:nid:opc:job ll_ost_io00_002:lustre-OST0000_UUID:6654:1876702022114816:192.168.203.48@tcp:104: 00010000:00010000:2.0:1789762593.257036:0:6654:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01697 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762593.257176:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762593.257355:0:6654:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01697 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.257364:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01697 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762593.399362:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead6e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022115456:0@lo:400:kworker.0 00000100:00100000:0.0:1789762593.399478:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d84542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022115328:0@lo:400:kworker.0 00000100:00100000:0.0:1789762593.399554:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762593.399632:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022115328 00000100:00100000:0.0:1789762593.399648:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022115328 00000100:00100000:0.0:1789762593.399723:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 38 00000100:00100000:0.0:1789762593.399739:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022115328:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762593.399868:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022115328:12345-0@lo:400:kworker.0 Request processed in 129us (312us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762593.399947:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 38 00000100:00100000:2.0:1789762593.400210:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022115584 00000100:00080000:0.0:1789762593.400223:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762593.400226:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022115584 00000100:00080000:0.0:1789762593.400236:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762593.400303:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d84542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022115328:0@lo:400:kworker.0 00000100:00100000:2.0:1789762593.400317:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 26 00000100:00100000:2.0:1789762593.400346:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022115584:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762593.400383:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762593.400463:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022115584:12345-0@lo:400:kworker.0 Request processed in 112us (752us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762593.400475:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 26 00000100:00080000:0.0:1789762593.400537:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762593.400686:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead6e6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022115456:0@lo:400:kworker.0 00000001:00080000:0.0:1789762593.414845:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967348 is committed 00000004:00080000:0.0:1789762593.414963:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 2 committed 00000004:00080000:0.0:1789762593.414993:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 3 committed 00000004:00080000:0.0:1789762593.415048:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 2 committed 00000100:00100000:0.0:1789762593.415240:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead4f5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022115840:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789762593.415385:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762593.415478:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 47 00000100:00100000:2.0:1789762593.415495:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022115712:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1789762593.415523:0:6649:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:2 count 1 00002000:00010000:2.0:1789762593.415562:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x2:0x0] initial LVB size: 0, mtime: 0x6aad9c1c, atime: 0x6aad9c1c, ctime: 0x6aad9c1c, blocks: 0x1 00010000:00010000:2.0:1789762593.415581:0:6649:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0169e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.415611:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2:0x0].0x0 since=116s v=(0 0) 00010000:00010000:2.0:1789762593.415621:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x2:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762593.415639:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0169e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.415706:0:6649:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0169e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.415732:0:6649:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0169e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.415748:0:6649:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0169e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.415774:0:6649:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1789762593.415847:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762593.415862:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0169e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.415893:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762593.415944:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0169e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.415966:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0169e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.416017:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0169e lrc: 1/0,0 mode: --/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.416034:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0169e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.416068:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762593.416073:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762593.416076:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762593.416877:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022115712:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1384us (1564us total) trans 4294967325 rc 0/0 00000100:00100000:2.0:1789762593.416893:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 47 00000004:00080000:0.0:1789762593.417010:0:3274:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ead4f5f80/2, rc 0, transno 4294967320 00000100:00100000:0.0:1789762593.417032:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022115840:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789762593.488184:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762593.656479:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 116960305346 time 116960382072 state 0x0 ref 1 00000100:00100000:0.0:1789762593.656508:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014774784:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762593.656559:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 2097152, mtime: 0x6aad9c21, atime: 0x6aad9c21, ctime: 0x6aad9c21, blocks: 0x1007 00010000:00010000:0.0:1789762593.656566:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b016b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.656579:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=116s v=(0 0) 00010000:00010000:0.0:1789762593.656583:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762593.656587:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b016b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762593.656599:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b016b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762593.719245:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762593.868250:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967328 is committed 00000100:00100000:0.0:1789762593.870670:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:0.0:1789762594.056663:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789762594.067699:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967329 is committed 00000100:00100000:0.0:1789762594.070218:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762594.268314:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967330 is committed 00000100:00100000:0.0:1789762594.270537:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762594.470010:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967331 is committed 00000100:00100000:0.0:1789762594.472535:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762594.669581:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967332 is committed 00000100:00100000:0.0:1789762594.671783:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762594.743168:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762594.856976:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 118160790562 time 118160877909 state 0x0 ref 1 00000100:00100000:2.0:1789762594.857005:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702014775552:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762594.857053:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 8388608, mtime: 0x6aad9c22, atime: 0x6aad9c21, ctime: 0x6aad9c22, blocks: 0x4007 00010000:00010000:2.0:1789762594.857061:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000597aded0/0x416193e012b016dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762594.857074:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=118s v=(0 0) 00010000:00010000:2.0:1789762594.857079:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762594.857133:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000597aded0/0x416193e012b016dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762594.857145:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000597aded0/0x416193e012b016dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762594.870935:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967333 is committed 00000001:00080000:2.0:1789762595.069533:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967334 is committed 00000100:00100000:0.0:1789762595.071777:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762595.270375:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967335 is committed 00000100:00100000:0.0:1789762595.272865:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762595.472257:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967336 is committed 00000100:00100000:0.0:1789762595.474928:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762595.657409:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 118961156774 time 118961310912 state 0x0 ref 1 00000100:00100000:2.0:1789762595.657437:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702014776064:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762595.657487:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 12582912, mtime: 0x6aad9c23, atime: 0x6aad9c21, ctime: 0x6aad9c23, blocks: 0x6007 00010000:00010000:2.0:1789762595.657495:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b016f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762595.657508:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=118s v=(0 0) 00010000:00010000:2.0:1789762595.657512:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762595.657516:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b016f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762595.657528:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b016f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762595.669704:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967337 is committed 00000001:00080000:0.0:1789762595.870876:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967338 is committed 00000100:00080000:0.0:1789762596.057619:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789762596.068915:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967339 is committed 00000100:00100000:0.0:1789762596.071247:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762596.257635:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 119561460729 time 119561538161 state 0x0 ref 1 00000100:00100000:2.0:1789762596.257663:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702014776448:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762596.257715:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 15728640, mtime: 0x6aad9c24, atime: 0x6aad9c21, ctime: 0x6aad9c24, blocks: 0x7807 00010000:00010000:2.0:1789762596.257734:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b0170e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762596.257747:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=119s v=(0 0) 00010000:00010000:2.0:1789762596.257751:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762596.257755:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b0170e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762596.257767:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b0170e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762596.269811:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967340 is committed 00000100:00100000:0.0:1789762596.272506:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762596.457752:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 119761538161 time 119761651179 state 0x0 ref 1 00000100:00100000:2.0:1789762596.457785:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014776576:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762596.457849:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 16777216, mtime: 0x6aad9c24, atime: 0x6aad9c21, ctime: 0x6aad9c24, blocks: 0x8007 00010000:00010000:2.0:1789762596.457860:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000597aded0/0x416193e012b01715 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762596.457879:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=119s v=(0 0) 00010000:00010000:2.0:1789762596.457903:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762596.457912:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000597aded0/0x416193e012b01715 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762596.457930:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000597aded0/0x416193e012b01715 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762596.469992:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967341 is committed 00000001:00080000:2.0:1789762596.670630:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967342 is committed 00000100:00100000:0.0:1789762596.672939:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762596.791209:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762596.872726:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967343 is committed 00000100:00100000:0.0:1789762596.875391:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762597.081048:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967344 is committed 00000100:00100000:2.0:1789762597.084176:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762597.258214:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 120561973419 time 120562108306 state 0x0 ref 1 00000100:00100000:2.0:1789762597.258256:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702014777088:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762597.258330:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 20971520, mtime: 0x6aad9c25, atime: 0x6aad9c21, ctime: 0x6aad9c25, blocks: 0xa007 00010000:00010000:2.0:1789762597.258343:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01731 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762597.258365:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=120s v=(0 0) 00010000:00010000:2.0:1789762597.258371:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762597.258379:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01731 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762597.258398:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01731 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762597.282306:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967345 is committed 00000100:00100000:2.0:1789762597.284993:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762597.480450:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967346 is committed 00000100:00100000:2.0:1789762597.483391:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762597.658497:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 120962227438 time 120962391106 state 0x0 ref 1 00000100:00100000:2.0:1789762597.658552:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702014777344:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762597.658629:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 23068672, mtime: 0x6aad9c25, atime: 0x6aad9c21, ctime: 0x6aad9c25, blocks: 0xb007 00010000:00010000:2.0:1789762597.658641:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a66c43a3/0x416193e012b0173f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762597.658682:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=120s v=(0 0) 00010000:00010000:2.0:1789762597.658688:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762597.658697:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a66c43a3/0x416193e012b0173f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762597.658715:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a66c43a3/0x416193e012b0173f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762597.681367:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967347 is committed 00000100:00100000:2.0:1789762597.684059:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762597.815224:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762597.870626:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967348 is committed 00000100:00100000:0.0:1789762597.872730:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762597.986333:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762597.986365:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762597.986388:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762597.986419:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762597.986449:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014777856 00000100:00100000:2.0:1789762597.986455:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014777856 00000100:00100000:0.0:1789762597.986457:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014778112 00000100:00100000:0.0:1789762597.986464:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014778112 00000100:00100000:2.0:1789762597.986505:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 40 00000100:00100000:0.0:1789762597.986521:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:2.0:1789762597.986523:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702014777856:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762597.986540:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702014778112:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762597.986583:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702014777856:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 62us (189us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762597.986590:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702014778112:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 53us (169us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762597.986606:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 40 00000100:00100000:0.0:1789762597.986616:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:0.0:1789762597.989063:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762597.989141:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762597.989182:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 49 00000100:00100000:0.0:1789762597.989194:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2362:x1876702014778240:12345-192.168.203.48@tcp:9:kworker.0 00000020:00000080:0.0:1789762597.989219:0:6881:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x416193e012b010d2 00000100:00080000:0.0:1789762597.989249:0:6881:0:(import.c:65:import_set_state_nolock()) ffff981ebc386000 : changing import state from FULL to CLOSED 00000001:00000004:0.0:1789762597.989291:0:6881:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294967348 00010000:00010000:2.0:1789762597.989301:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1789762597.989327:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762597.989688:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+2:2362:x1876702014778240:12345-192.168.203.48@tcp:9:kworker.0 Request processed in 496us (624us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762597.989704:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 49 00000004:00080000:2.0:1789762598.007213:0:6868:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 121311124204ns, next at 122311121141ns, rc = 0 00000100:00100000:0.0:1789762598.007303:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022115968 00000100:00100000:0.0:1789762598.007309:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022115968 00000100:00080000:2.0:1789762598.007351:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762598.007359:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 21 00000100:00100000:0.0:1789762598.007368:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022115968:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762598.007383:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 112394240 pending 0 free 3818913792 avail 3815768064 00000100:00100000:0.0:1789762598.007420:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022115968:12345-0@lo:13:osp-pre-0-0.0 Request processed in 55us (143us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762598.007433:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 21 00000100:00100000:2.0:1789762598.058661:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 121362480681 time 121362561049 state 0x0 ref 1 00000100:00100000:2.0:1789762598.058710:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702014777600:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762598.058759:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 25165824, mtime: 0x6aad9c25, atime: 0x6aad9c21, ctime: 0x6aad9c25, blocks: 0xc007 00010000:00010000:2.0:1789762598.058767:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009326100a/0x416193e012b0174d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.058781:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=121s v=(0 0) 00010000:00010000:2.0:1789762598.058785:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762598.058797:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009326100a/0x416193e012b0174d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.058817:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009326100a/0x416193e012b0174d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00100000:0.0:1789762598.070403:0:6674:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 871a7a42-3659-47e5-8fc9-f4f1e262590d 00000020:00000080:0.0:1789762598.070410:0:6674:0:(genops.c:885:class_export_put()) final put ffff981ebc384800/871a7a42-3659-47e5-8fc9-f4f1e262590d 00000001:00080000:0.0:1789762598.070432:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967349 is committed 00000001:00080000:0.0:1789762598.273935:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967350 is committed 00000100:00100000:2.0:1789762598.276496:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762598.458938:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 121762753621 time 121762839656 state 0x0 ref 1 00000100:00100000:2.0:1789762598.458965:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014778624:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762598.459014:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 27262976, mtime: 0x6aad9c26, atime: 0x6aad9c21, ctime: 0x6aad9c26, blocks: 0xd007 00010000:00010000:2.0:1789762598.459021:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b0175b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.459035:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=121s v=(0 0) 00010000:00010000:2.0:1789762598.459038:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762598.459043:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b0175b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.459071:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b0175b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762598.470699:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:0.0:1789762598.470707:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:0.0:1789762598.470718:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b0175b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762598.470728:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b0175b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762598.470735:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b0175b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762598.470740:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e1a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762598.470743:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b0175b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762598.470750:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b0175b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762598.470764:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b0175b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762598.470769:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b0175b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762598.470841:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014778624:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11885us (194344us total) trans 4294967351 rc 0/0 00000100:00100000:0.0:1789762598.470849:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 61 00000100:00100000:0.0:1789762598.470855:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 121762753621 time 121762839656 state 0x2 ref 0 00000004:00080000:2.0:1789762598.519278:0:7726:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 121823171963ns, next at 122823167303ns, rc = 0 00000100:00100000:0.0:1789762598.519279:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc01180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022116096:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762598.519313:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022116480:0@lo:400:kworker.0 00000100:00100000:0.0:1789762598.519348:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762598.519377:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762598.519418:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022116608:0@lo:400:kworker.0 00000100:00100000:2.0:1789762598.519448:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022116352:0@lo:400:kworker.0 00000100:00100000:0.0:1789762598.519453:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762598.519518:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762598.519520:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022116096 00000100:00100000:0.0:1789762598.519528:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022116096 00000100:00100000:0.0:1789762598.519585:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 22 00000100:00100000:0.0:1789762598.519596:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82730380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022116096:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789762598.519644:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3851419648 avail 3849322496 00000100:00100000:0.0:1789762598.519709:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82730380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022116096:12345-0@lo:13:osp-pre-1-0.0 Request processed in 115us (360us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762598.519717:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 22 00000100:00100000:2.0:1789762598.519752:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 43 00000100:00100000:0.0:1789762598.519761:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 44 00000100:00100000:2.0:1789762598.519766:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82733b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022116608:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762598.519782:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82087b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022116352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762598.519819:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82733b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022116608:12345-0@lo:400:kworker.0 Request processed in 56us (367us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762598.519829:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82087b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022116352:12345-0@lo:400:kworker.0 Request processed in 57us (315us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762598.519831:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 43 00000100:00100000:0.0:1789762598.519842:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 44 00000100:00100000:2.0:1789762598.519890:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022116352:0@lo:400:kworker.0 00000100:00100000:0.0:1789762598.519916:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022116608:0@lo:400:kworker.0 00000100:00080000:2.0:1789762598.519931:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762598.519937:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762598.519946:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc01180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022116096:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762598.519971:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022116480:0@lo:400:kworker.0 00000004:00080000:0.0:1789762598.527431:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 4 committed 00000100:00100000:0.0:1789762598.527523:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82086d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022116864:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789762598.527574:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762598.527654:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022116992 00000100:00100000:2.0:1789762598.527661:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022116992 00000100:00100000:2.0:1789762598.527701:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:2.0:1789762598.527715:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022116992:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1789762598.527732:0:10916:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:3 count 1 00010000:00010000:2.0:1789762598.527747:0:10916:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b01769 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.527782:0:10916:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007c0fd336/0x416193e012b014ec lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd043 expref: 8 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.527799:0:10916:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b014f3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd03c expref: 8 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.527815:0:10916:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003cb1c4b8/0x416193e012b014e5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd035 expref: 8 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.527825:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=5s v=(0 0) 00010000:00010000:2.0:1789762598.527828:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762598.527833:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b01769 lrc: 6/0,1 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.527865:0:10916:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003cb1c4b8/0x416193e012b014e5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd035 expref: 8 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.527894:0:10916:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003cb1c4b8/0x416193e012b014e5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd035 expref: 8 pid: 6647 timeout: 221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762598.527907:0:10916:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82086a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10916:1876702022117120:192.168.203.48@tcp:104: 00010000:00010000:0.0:1789762598.527944:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762598.527950:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01762 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762598.527960:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa64e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762598.527964:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01762 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762598.527991:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01762 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.528002:0:10916:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b014f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd03c expref: 8 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762598.528009:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01762 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762598.528027:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01762 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.528054:0:10916:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b014f3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd03c expref: 8 pid: 10916 timeout: 221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762598.528076:0:10916:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82086680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10916:1876702022117248:192.168.203.48@tcp:104: 00010000:00010000:0.0:1789762598.528080:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762598.528086:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762598.528089:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762598.528168:0:10916:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007c0fd336/0x416193e012b014ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd043 expref: 8 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.528178:0:10916:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007c0fd336/0x416193e012b014ec lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd043 expref: 8 pid: 6881 timeout: 221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762598.528186:0:10916:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82085180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10916:1876702022117376:192.168.203.48@tcp:104: 00000100:00100000:2.0:1789762598.528298:0:10916:0:(client.c:2715:ptlrpc_set_wait()) set ffff981eb450bf00 going to sleep for 11 seconds 00000004:00080000:2.0:1789762598.528682:0:3274:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981d82086d80/2, rc 0, transno 4294967352 00000100:00100000:2.0:1789762598.528691:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82086d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022116864:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762598.529231:0:10916:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82086a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10916:1876702022117120:192.168.203.48@tcp:104: 00000100:00100000:2.0:1789762598.529342:0:10916:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82086680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10916:1876702022117248:192.168.203.48@tcp:104: 00000100:00100000:2.0:1789762598.529356:0:10916:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82085180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10916:1876702022117376:192.168.203.48@tcp:104: 00010000:00010000:2.0:1789762598.529374:0:10916:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b01769 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762598.529474:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014779136 00000100:00100000:2.0:1789762598.529482:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014779136 00010000:00010000:2.0:1789762598.529515:0:6664:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007c0fd336/0x416193e012b014ec lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd043 expref: 10 pid: 6881 timeout: 221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762598.529564:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014779008 00000100:00100000:2.0:1789762598.529567:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014779008 00010000:00010000:2.0:1789762598.529582:0:6664:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b014f3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd03c expref: 10 pid: 10916 timeout: 221 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762598.529614:0:6664:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 38 00000100:00100000:2.0:1789762598.529631:0:6664:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014779008:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762598.529649:0:6664:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762598.529660:0:6664:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b014f3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd03c expref: 9 pid: 10916 timeout: 221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.529692:0:6664:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b014f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x3:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd03c expref: 8 pid: 10916 timeout: 221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.529712:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b014f3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd03c expref: 8 pid: 10916 timeout: 221 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.529725:0:6664:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762598.529731:0:6664:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x3:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762598.529740:0:6664:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b01769 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.529751:0:6664:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b01769 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.529780:0:6664:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762598.529805:0:6664:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014779008:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 181us (324us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762598.529819:0:6664:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 38 00010000:00010000:2.0:1789762598.529888:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762598.529893:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b01769 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.529900:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e1e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762598.529904:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b01769 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.529910:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b01769 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.529920:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b01769 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.529925:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b01769 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762598.529936:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762598.529937:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762598.529938:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762598.672758:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967354 is committed 00000001:00080000:2.0:1789762598.672778:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967353 is committed 00000100:00100000:0.0:1789762598.675573:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014779264 00000100:00100000:0.0:1789762598.675584:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014779264 00000100:00100000:0.0:1789762598.675627:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 121962839656 time 121962904164 state 0x0 ref 1 00000100:00100000:0.0:1789762598.675638:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 122162904164 time 121962904164 state 0x1 ref 1 00010000:00010000:2.0:1789762598.839205:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762599.071123:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967356 is committed 00000100:00100000:0.0:1789762599.073584:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762599.259268:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 122563035704 time 122563170358 state 0x0 ref 1 00000100:00100000:0.0:1789762599.259297:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82733800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014779520:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762599.259348:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 31457280, mtime: 0x6aad9c27, atime: 0x6aad9c21, ctime: 0x6aad9c27, blocks: 0xf007 00010000:00010000:0.0:1789762599.259356:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b01785 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762599.259369:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=122s v=(0 0) 00010000:00010000:0.0:1789762599.259373:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762599.259378:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b01785 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762599.259389:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b01785 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762599.270638:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967357 is committed 00000100:00100000:0.0:1789762599.273240:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762599.470208:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967358 is committed 00000100:00100000:2.0:1789762599.472571:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014779776 00000100:00100000:2.0:1789762599.472579:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014779776 00000100:00100000:2.0:1789762599.472610:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 122763170358 time 122763263768 state 0x0 ref 1 00000100:00100000:2.0:1789762599.472619:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 122963263768 time 122763263768 state 0x1 ref 1 00000001:00080000:0.0:1789762599.672584:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967359 is committed 00010000:00010000:0.0:1789762599.863206:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762599.870847:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967360 is committed 00000100:00100000:0.0:1789762599.873151:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762600.059605:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 123363410496 time 123363509551 state 0x0 ref 1 00000100:00100000:0.0:1789762600.059638:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82732680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014780032:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762600.059689:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 35651584, mtime: 0x6aad9c27, atime: 0x6aad9c21, ctime: 0x6aad9c27, blocks: 0x11007 00010000:00010000:0.0:1789762600.059699:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b017a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762600.059720:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=123s v=(0 0) 00010000:00010000:0.0:1789762600.059724:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762600.059730:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b017a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762600.059742:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b017a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762600.071395:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967361 is committed 00000100:00100000:0.0:1789762600.073713:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762600.272002:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967362 is committed 00000100:00100000:2.0:1789762600.274384:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014780288 00000100:00100000:2.0:1789762600.274395:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014780288 00000100:00100000:2.0:1789762600.274431:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 123563509551 time 123563593543 state 0x0 ref 1 00000100:00100000:2.0:1789762600.274441:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 123763593543 time 123563593543 state 0x1 ref 1 00000100:00100000:2.0:1789762600.459754:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 123763593543 time 123763656531 state 0x0 ref 1 00000100:00100000:2.0:1789762600.459782:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014780288:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762600.459831:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 37748736, mtime: 0x6aad9c28, atime: 0x6aad9c21, ctime: 0x6aad9c28, blocks: 0x12007 00010000:00010000:2.0:1789762600.459839:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b017af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762600.459865:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=123s v=(0 0) 00010000:00010000:2.0:1789762600.459869:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762600.459874:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b017af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762600.459885:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b017af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762600.473008:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762600.674228:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762600.859919:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 124163718921 time 124163821674 state 0x0 ref 1 00000100:00100000:2.0:1789762600.859946:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82730e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014780544:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762600.859996:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 39845888, mtime: 0x6aad9c28, atime: 0x6aad9c21, ctime: 0x6aad9c28, blocks: 0x13007 00010000:00010000:2.0:1789762600.860004:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000198d2894/0x416193e012b017bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762600.860016:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=124s v=(0 0) 00010000:00010000:2.0:1789762600.860020:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762600.860024:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000198d2894/0x416193e012b017bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762600.860037:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000198d2894/0x416193e012b017bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762600.874421:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762600.887156:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762601.075274:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014780800 00000100:00100000:2.0:1789762601.075282:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014780800 00000100:00100000:2.0:1789762601.075320:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 124363821674 time 124363904779 state 0x0 ref 1 00000100:00100000:2.0:1789762601.075338:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 124563904779 time 124363904779 state 0x1 ref 1 00000100:00100000:2.0:1789762601.260053:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 124563904779 time 124563955629 state 0x0 ref 1 00000100:00100000:2.0:1789762601.260081:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014780800:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762601.260146:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 41943040, mtime: 0x6aad9c29, atime: 0x6aad9c21, ctime: 0x6aad9c29, blocks: 0x14007 00010000:00010000:2.0:1789762601.260166:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b017cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762601.260179:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=124s v=(0 0) 00010000:00010000:2.0:1789762601.260182:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762601.260187:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b017cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762601.260199:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b017cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762601.273702:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762601.474864:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762601.660199:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 124964019983 time 124964101981 state 0x0 ref 1 00000100:00100000:2.0:1789762601.660224:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014781056:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762601.660274:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 44040192, mtime: 0x6aad9c29, atime: 0x6aad9c21, ctime: 0x6aad9c29, blocks: 0x15007 00010000:00010000:2.0:1789762601.660281:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009326100a/0x416193e012b017d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762601.660294:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=124s v=(0 0) 00010000:00010000:2.0:1789762601.660298:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762601.660303:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009326100a/0x416193e012b017d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762601.660313:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009326100a/0x416193e012b017d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762601.674894:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762601.872284:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967370 is committed 00000100:00100000:2.0:1789762601.874756:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762601.911234:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762602.060398:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 125364196256 time 125364301441 state 0x0 ref 1 00000100:00100000:2.0:1789762602.060435:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82086680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014781312:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762602.060485:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 46137344, mtime: 0x6aad9c29, atime: 0x6aad9c21, ctime: 0x6aad9c29, blocks: 0x16009 00010000:00010000:2.0:1789762602.060496:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a66c43a3/0x416193e012b017e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762602.060513:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=125s v=(0 0) 00010000:00010000:2.0:1789762602.060518:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762602.060525:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a66c43a3/0x416193e012b017e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762602.060541:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a66c43a3/0x416193e012b017e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762602.072237:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967371 is committed 00000001:00080000:2.0:1789762602.272876:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967372 is committed 00000100:00100000:2.0:1789762602.275861:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014781568 00000100:00100000:2.0:1789762602.275869:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014781568 00000100:00100000:2.0:1789762602.275903:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 125564301441 time 125564361924 state 0x0 ref 1 00000100:00100000:2.0:1789762602.275912:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 125764361924 time 125564361924 state 0x1 ref 1 00000001:00080000:2.0:1789762602.473168:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967373 is committed 00000001:00080000:2.0:1789762602.673984:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967374 is committed 00000100:00100000:2.0:1789762602.860724:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 126164545411 time 126164626661 state 0x0 ref 1 00000100:00100000:2.0:1789762602.860748:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014781824:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762602.860807:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 50331648, mtime: 0x6aad9c2a, atime: 0x6aad9c21, ctime: 0x6aad9c2a, blocks: 0x18009 00010000:00010000:2.0:1789762602.860815:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01803 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762602.860828:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=126s v=(0 0) 00010000:00010000:2.0:1789762602.860832:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762602.860848:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762602.860861:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762602.873186:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 50331648 -> 51380224 00002000:00010000:0.0:1789762602.873193:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 98313 -> 100361 00010000:00010000:0.0:1789762602.873206:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01803 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762602.873221:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01803 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762602.873231:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01803 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762602.873248:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762602.873254:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01803 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762602.873266:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01803 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762602.873286:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01803 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762602.873295:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b01803 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762602.873372:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014781824:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12630us (196831us total) trans 4294967375 rc 0/0 00000100:00100000:0.0:1789762602.873384:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 83 00000100:00100000:0.0:1789762602.873393:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 126164545411 time 126164626661 state 0x2 ref 0 00000100:00100000:0.0:1789762602.875615:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762602.935205:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762603.075233:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762603.075287:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014782080 00000100:00100000:2.0:1789762603.075296:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014782080 00000100:00100000:2.0:1789762603.075340:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 126364626661 time 126364702993 state 0x0 ref 1 00000100:00100000:2.0:1789762603.075352:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 126564702993 time 126364702993 state 0x1 ref 1 00000100:00100000:2.0:1789762603.106131:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762603.106157:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762603.106176:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762603.106222:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762603.106229:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014782464 00000100:00100000:0.0:1789762603.106237:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014782464 00000100:00100000:2.0:1789762603.106265:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014782592 00000100:00100000:2.0:1789762603.106271:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014782592 00000100:00100000:0.0:1789762603.106295:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 29 00000100:00100000:0.0:1789762603.106311:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82087100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014782464:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762603.106324:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 46 00000100:00100000:2.0:1789762603.106338:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82084e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702014782592:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762603.106389:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82087100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014782464:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 78us (203us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762603.106390:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82084e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702014782592:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 55us (169us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762603.106419:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 46 00000100:00100000:0.0:1789762603.106420:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 29 00000100:00080000:0.0:1789762603.106497:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762603.106505:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762603.127339:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022117504 00000100:00100000:0.0:1789762603.127347:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022117504 00000100:00080000:2.0:1789762603.127388:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762603.127405:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23 00000100:00100000:0.0:1789762603.127416:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022117504:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762603.127434:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 156696576 pending 0 free 3814719488 avail 3805282304 00000100:00100000:0.0:1789762603.127491:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022117504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762603.127501:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23 00000100:00100000:2.0:1789762603.260868:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 126564702993 time 126564770773 state 0x0 ref 1 00000100:00100000:2.0:1789762603.260892:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82733480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014782080:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762603.260939:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 52428800, mtime: 0x6aad9c2b, atime: 0x6aad9c21, ctime: 0x6aad9c2b, blocks: 0x19009 00010000:00010000:2.0:1789762603.260946:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b01811 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762603.260958:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=126s v=(0 0) 00010000:00010000:2.0:1789762603.260962:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762603.260967:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b01811 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762603.260978:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b01811 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762603.276562:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000004:00080000:2.0:1789762603.639234:0:7726:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 126943144322ns, next at 127943141165ns, rc = 0 00000100:00100000:0.0:1789762603.639245:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82086d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022117760:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762603.639274:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead4f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022117888:0@lo:400:kworker.0 00000100:00100000:0.0:1789762603.639331:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762603.639332:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762603.639399:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead4f4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022118144:0@lo:400:kworker.0 00000100:00100000:2.0:1789762603.639415:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022117888 00000100:00100000:2.0:1789762603.639421:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022117888 00000100:00100000:0.0:1789762603.639431:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762603.639475:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022118144 00000100:00100000:2.0:1789762603.639480:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022118144 00000100:00100000:0.0:1789762603.639493:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022117760 00000100:00100000:0.0:1789762603.639498:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022117760 00000100:00100000:2.0:1789762603.639547:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022117632 00000100:00100000:0.0:1789762603.639551:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24 00000100:00100000:2.0:1789762603.639553:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022117632 00000100:00100000:0.0:1789762603.639562:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82731c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022117760:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789762603.639592:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3851419648 avail 3849322496 00000100:00100000:2.0:1789762603.639592:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 223 00000100:00100000:2.0:1789762603.639603:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022117632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762603.639642:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82731c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022117760:12345-0@lo:13:osp-pre-1-0.0 Request processed in 83us (318us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762603.639649:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24 00000100:00100000:2.0:1789762603.639657:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022117632:12345-0@lo:400:kworker.0 Request processed in 56us (236us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762603.639665:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 223 00000100:00100000:0.0:1789762603.639699:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82086d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022117760:0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1789762603.639701:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762603.639778:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762603.639784:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762603.639806:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022117888:0@lo:400:kworker.0 00000100:00100000:2.0:1789762603.639853:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022118016 00000100:00100000:0.0:1789762603.639858:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022118144:0@lo:400:kworker.0 00000100:00100000:2.0:1789762603.639862:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022118016 00000100:00100000:2.0:1789762603.639905:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 49 00000100:00100000:2.0:1789762603.639915:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022118016:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762603.639919:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022117632:0@lo:400:kworker.0 00000100:00100000:0.0:1789762603.639929:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022118272:0@lo:400:kworker.0 00000100:00100000:2.0:1789762603.639961:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022118016:12345-0@lo:400:kworker.0 Request processed in 47us (178us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762603.639967:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 49 00000100:00080000:2.0:1789762603.639984:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00080000:0.0:1789762603.646536:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 3 committed 00000100:00100000:2.0:1789762603.646633:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022118400:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789762603.646672:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1789762603.646947:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762603.646954:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000829abbe0/0x416193e012b0181f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762603.646963:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa64800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762603.646967:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000829abbe0/0x416193e012b0181f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762603.646975:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000829abbe0/0x416193e012b0181f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762603.646991:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000829abbe0/0x416193e012b0181f lrc: 1/0,0 mode: --/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762603.646996:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000829abbe0/0x416193e012b0181f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x2:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762603.647008:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762603.647009:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762603.647010:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1789762603.647522:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981d8257e680/2, rc 0, transno 4294967321 00000100:00100000:2.0:1789762603.647530:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022118400:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789762603.661054:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 126964852533 time 126964956068 state 0x0 ref 1 00000100:00100000:2.0:1789762603.661126:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014782848:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762603.661192:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 54525952, mtime: 0x6aad9c2b, atime: 0x6aad9c21, ctime: 0x6aad9c2b, blocks: 0x1a009 00010000:00010000:2.0:1789762603.661199:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b01826 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762603.661213:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=126s v=(0 0) 00010000:00010000:2.0:1789762603.661223:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762603.661230:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b01826 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762603.661245:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b01826 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762603.673145:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967379 is committed 00000100:00100000:2.0:1789762603.875856:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014783104 00000100:00100000:2.0:1789762603.875865:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014783104 00000100:00100000:2.0:1789762603.875909:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 127164956068 time 127165043389 state 0x0 ref 1 00000100:00100000:2.0:1789762603.875922:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 127365043389 time 127165043389 state 0x1 ref 1 00010000:00010000:0.0:1789762603.959165:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762604.061203:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 127365043389 time 127365106098 state 0x0 ref 1 00000100:00100000:2.0:1789762604.061242:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014783104:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:0.0:1789762604.061268:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762604.061291:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 56623104, mtime: 0x6aad9c2b, atime: 0x6aad9c21, ctime: 0x6aad9c2b, blocks: 0x1b009 00010000:00010000:2.0:1789762604.061300:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01834 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762604.061331:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=127s v=(0 0) 00010000:00010000:2.0:1789762604.061334:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762604.061339:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762604.061353:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01834 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762604.076659:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762604.274035:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967382 is committed 00000100:00100000:2.0:1789762604.461347:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 127765170684 time 127765249761 state 0x0 ref 1 00000100:00100000:2.0:1789762604.461375:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014783360:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762604.461423:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 58720256, mtime: 0x6aad9c2c, atime: 0x6aad9c21, ctime: 0x6aad9c2c, blocks: 0x1c009 00010000:00010000:2.0:1789762604.461430:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007990669e/0x416193e012b01842 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762604.461453:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=127s v=(0 0) 00010000:00010000:2.0:1789762604.461459:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762604.461465:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007990669e/0x416193e012b01842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762604.461476:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007990669e/0x416193e012b01842 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762604.472986:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967383 is committed 00000001:00080000:0.0:1789762604.674491:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967384 is committed 00000100:00100000:2.0:1789762604.676962:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014783616 00000100:00100000:2.0:1789762604.676978:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014783616 00000100:00100000:2.0:1789762604.677016:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 127965249761 time 127965333954 state 0x0 ref 1 00000100:00100000:2.0:1789762604.677026:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 128165333954 time 127965333954 state 0x1 ref 1 00000100:00100000:2.0:1789762604.861511:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 128165333954 time 128165412521 state 0x0 ref 1 00000100:00100000:2.0:1789762604.861546:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014783616:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762604.861603:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 60817408, mtime: 0x6aad9c2c, atime: 0x6aad9c21, ctime: 0x6aad9c2c, blocks: 0x1d009 00010000:00010000:2.0:1789762604.861613:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a66c43a3/0x416193e012b01850 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762604.861631:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=128s v=(0 0) 00010000:00010000:2.0:1789762604.861636:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762604.861643:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a66c43a3/0x416193e012b01850 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762604.861661:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a66c43a3/0x416193e012b01850 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762604.874776:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967385 is committed 00010000:00010000:2.0:1789762604.983221:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762605.073552:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967386 is committed 00000100:00100000:2.0:1789762605.075841:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014783872 00000100:00100000:2.0:1789762605.075850:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014783872 00000100:00100000:2.0:1789762605.075892:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 128365412521 time 128365470103 state 0x0 ref 1 00000100:00100000:2.0:1789762605.075903:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 128565470103 time 128365470103 state 0x1 ref 1 00000001:00080000:2.0:1789762605.188007:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967321 is committed 00000100:00100000:2.0:1789762605.261671:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 128565470103 time 128565566723 state 0x0 ref 1 00000100:00100000:2.0:1789762605.261728:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014783872:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762605.261797:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 62914560, mtime: 0x6aad9c2d, atime: 0x6aad9c21, ctime: 0x6aad9c2d, blocks: 0x1e009 00010000:00010000:2.0:1789762605.261806:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e808ba6/0x416193e012b0185e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762605.261842:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=128s v=(0 0) 00010000:00010000:2.0:1789762605.261847:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762605.261853:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006e808ba6/0x416193e012b0185e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762605.261867:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006e808ba6/0x416193e012b0185e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762605.278413:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967387 is committed 00002000:00010000:2.0:1789762605.278439:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:2.0:1789762605.278448:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 122889 -> 124937 00010000:00010000:2.0:1789762605.278464:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006e808ba6/0x416193e012b0185e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762605.278478:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006e808ba6/0x416193e012b0185e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762605.278486:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006e808ba6/0x416193e012b0185e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762605.278492:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e1200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762605.278496:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006e808ba6/0x416193e012b0185e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762605.278513:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006e808ba6/0x416193e012b0185e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762605.278531:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006e808ba6/0x416193e012b0185e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762605.278536:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006e808ba6/0x416193e012b0185e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762605.278622:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014783872:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16904us (202839us total) trans 4294967387 rc 0/0 00000100:00100000:2.0:1789762605.278636:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 95 00000100:00100000:2.0:1789762605.278646:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 128565470103 time 128565566723 state 0x2 ref 0 00000100:00100000:0.0:1789762605.281063:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762605.476107:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967388 is committed 00000100:00100000:0.0:1789762605.479083:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762605.479162:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014784128 00000100:00100000:2.0:1789762605.479172:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014784128 00000100:00100000:2.0:1789762605.479214:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 128765566723 time 128765633798 state 0x0 ref 1 00000100:00100000:2.0:1789762605.479225:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 128965633798 time 128765633798 state 0x1 ref 1 00000001:00080000:2.0:1789762605.674691:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967389 is committed 00002000:00010000:0.0:1789762605.874454:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 66060288 -> 67108864 00002000:00010000:0.0:1789762605.874462:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 129033 -> 131081 00010000:00010000:0.0:1789762605.874475:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b01873 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762605.874486:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b01873 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762605.874493:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b01873 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762605.874498:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762605.874502:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b01873 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762605.874508:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b01873 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762605.874524:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b01873 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762605.874537:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b01873 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762605.874614:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc424a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014784256:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12665us (197170us total) trans 4294967390 rc 0/0 00000100:00100000:0.0:1789762605.874623:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 98 00000100:00100000:0.0:1789762605.874630:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 129165719493 time 129165834373 state 0x2 ref 0 00010000:00010000:2.0:1789762606.007206:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762606.062021:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 129365834373 time 129365923979 state 0x0 ref 1 00000100:00100000:2.0:1789762606.062062:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc424000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014784384:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:0.0:1789762606.062113:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762606.062160:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 67108864, mtime: 0x6aad9c2d, atime: 0x6aad9c21, ctime: 0x6aad9c2d, blocks: 0x20009 00010000:00010000:2.0:1789762606.062169:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b0187a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762606.062181:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=129s v=(0 0) 00010000:00010000:2.0:1789762606.062185:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762606.062191:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b0187a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762606.062214:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b0187a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762606.076745:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014784512 00000100:00100000:0.0:1789762606.076754:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014784512 00000100:00100000:0.0:1789762606.076788:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 129365834373 time 129365923979 state 0x0 ref 1 00000100:00100000:0.0:1789762606.076797:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 129565923979 time 129365923979 state 0x1 ref 1 00000100:00100000:2.0:1789762606.278829:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014784640 00000100:00100000:2.0:1789762606.278838:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014784640 00000100:00100000:2.0:1789762606.278874:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 129565923979 time 129566027536 state 0x0 ref 1 00000100:00100000:2.0:1789762606.278883:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 129766027536 time 129566027536 state 0x1 ref 1 00000100:00100000:0.0:1789762606.662297:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 129966121163 time 129966199301 state 0x0 ref 1 00000100:00100000:0.0:1789762606.662326:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc427480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014784768:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762606.662377:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 70254592, mtime: 0x6aad9c2e, atime: 0x6aad9c21, ctime: 0x6aad9c2e, blocks: 0x21809 00010000:00010000:0.0:1789762606.662384:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b0188f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762606.662397:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=129s v=(0 0) 00010000:00010000:0.0:1789762606.662401:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762606.662406:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b0188f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762606.662435:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b0188f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762606.862426:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 130166199301 time 130166325244 state 0x0 ref 1 00000100:00100000:2.0:1789762606.862459:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc424700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014784896:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762606.862518:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 71303168, mtime: 0x6aad9c2e, atime: 0x6aad9c21, ctime: 0x6aad9c2e, blocks: 0x22009 00010000:00010000:2.0:1789762606.862526:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b01896 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762606.862540:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=130s v=(0 0) 00010000:00010000:2.0:1789762606.862543:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762606.862561:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b01896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762606.862574:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b01896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762606.874331:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 71303168 -> 72351744 00002000:00010000:2.0:1789762606.874336:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 139273 -> 141321 00010000:00010000:2.0:1789762606.874349:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b01896 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762606.874364:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b01896 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762606.874376:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b01896 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762606.874384:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762606.874390:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b01896 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762606.874399:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b01896 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762606.874419:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b01896 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762606.874441:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b01896 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762606.874515:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc424700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014784896:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12070us (196346us total) trans 4294967395 rc 0/0 00000100:00100000:2.0:1789762606.874525:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 103 00000100:00100000:2.0:1789762606.874531:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 130166199301 time 130166325244 state 0x2 ref 0 00000100:00100000:0.0:1789762606.877241:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014785024 00000100:00100000:0.0:1789762606.877249:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014785024 00000100:00100000:0.0:1789762606.877284:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 130166199301 time 130166325244 state 0x0 ref 1 00000100:00100000:0.0:1789762606.877293:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 130366325244 time 130166325244 state 0x1 ref 1 00010000:00010000:0.0:1789762607.031235:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762607.077991:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762607.078034:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014785152 00000100:00100000:2.0:1789762607.078043:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014785152 00000100:00100000:2.0:1789762607.078081:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 130366325244 time 130366410673 state 0x0 ref 1 00000100:00100000:2.0:1789762607.078114:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 130566410673 time 130366410673 state 0x1 ref 1 00000001:00080000:2.0:1789762607.277323:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967397 is committed 00000100:00100000:0.0:1789762607.280113:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762607.462667:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 130766496468 time 130766568951 state 0x0 ref 1 00000100:00100000:0.0:1789762607.462693:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014785280:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762607.462743:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 74448896, mtime: 0x6aad9c2f, atime: 0x6aad9c21, ctime: 0x6aad9c2f, blocks: 0x23809 00010000:00010000:0.0:1789762607.462751:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b018ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762607.462764:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=130s v=(0 0) 00010000:00010000:0.0:1789762607.462769:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762607.462773:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b018ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762607.462784:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b018ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762607.662769:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 130966568951 time 130966668791 state 0x0 ref 1 00000100:00100000:2.0:1789762607.662817:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc425f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014785408:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762607.662877:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 75497472, mtime: 0x6aad9c2f, atime: 0x6aad9c21, ctime: 0x6aad9c2f, blocks: 0x24009 00010000:00010000:2.0:1789762607.662887:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b018b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762607.662905:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=130s v=(0 0) 00010000:00010000:2.0:1789762607.662910:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762607.662916:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b018b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762607.662949:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee05f9ab/0x416193e012b018b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762607.675775:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967399 is committed 00000001:00080000:2.0:1789762607.875466:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967400 is committed 00000100:00100000:2.0:1789762607.878386:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014785664 00000100:00100000:2.0:1789762607.878394:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014785664 00000100:00100000:2.0:1789762607.878432:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 131166668791 time 131166783158 state 0x0 ref 1 00000100:00100000:2.0:1789762607.878442:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 131366783158 time 131166783158 state 0x1 ref 1 00010000:00010000:0.0:1789762608.055174:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789762608.063022:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789762608.074946:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967401 is committed 00000100:00100000:0.0:1789762608.226368:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762608.226421:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762608.226452:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014786048 00000100:00100000:2.0:1789762608.226464:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014786048 00000100:00100000:0.0:1789762608.226474:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762608.226547:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 50 00000100:00100000:0.0:1789762608.226549:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762608.226568:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702014786048:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762608.226592:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014786304 00000100:00100000:0.0:1789762608.226600:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014786304 00000100:00100000:0.0:1789762608.226649:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 51 00000100:00100000:2.0:1789762608.226661:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702014786048:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 91us (292us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762608.226662:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702014786304:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762608.226686:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 50 00000100:00100000:0.0:1789762608.226716:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702014786304:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 56us (171us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762608.226727:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 51 00000100:00100000:0.0:1789762608.247380:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022118528 00000100:00100000:0.0:1789762608.247389:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022118528 00000100:00100000:0.0:1789762608.247461:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 25 00000100:00100000:0.0:1789762608.247473:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022118528:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762608.247495:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 199294976 pending 0 free 3783262208 avail 3779067904 00000100:00100000:0.0:1789762608.247550:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022118528:12345-0@lo:13:osp-pre-0-0.0 Request processed in 79us (235us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762608.247559:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 25 00000001:00080000:0.0:1789762608.275571:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967402 is committed 00000100:00100000:2.0:1789762608.463158:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 131766915016 time 131767058949 state 0x0 ref 1 00000100:00100000:2.0:1789762608.463191:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc426d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014786432:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762608.463246:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 79691776, mtime: 0x6aad9c30, atime: 0x6aad9c21, ctime: 0x6aad9c30, blocks: 0x26009 00010000:00010000:2.0:1789762608.463254:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b018ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762608.463268:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=131s v=(0 0) 00010000:00010000:2.0:1789762608.463291:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762608.463296:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b018ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762608.463307:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d70b1c8d/0x416193e012b018ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762608.474706:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967403 is committed 00000001:00080000:0.0:1789762608.675337:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967404 is committed 00000100:00100000:0.0:1789762608.677860:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762608.759241:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead4f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022118656:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762608.759316:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762608.759405:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82085f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022119296:0@lo:400:kworker.0 00000100:00100000:0.0:1789762608.759441:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762608.759490:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022118656 00000100:00100000:0.0:1789762608.759496:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022118656 00000100:00100000:2.0:1789762608.759559:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022119040 00000100:00100000:0.0:1789762608.759561:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 26 00000100:00100000:2.0:1789762608.759567:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022119040 00000100:00100000:0.0:1789762608.759574:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022118656:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789762608.759610:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762608.759613:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:0.0:1789762608.759666:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022118656:12345-0@lo:13:osp-pre-1-0.0 Request processed in 96us (352us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762608.759672:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022119040:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762608.759680:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 26 00000100:00100000:2.0:1789762608.759751:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022119040:12345-0@lo:400:kworker.0 Request processed in 85us (433us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762608.759766:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82085f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022119296:0@lo:400:kworker.0 00000100:00100000:2.0:1789762608.759769:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000100:00100000:0.0:1789762608.759783:0:3274:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967321 gen 2 00000004:00080000:0.0:1789762608.759808:0:3274:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981d8257e680, transno 4294967321 00000100:00100000:0.0:1789762608.759827:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022118656:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762608.759871:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762608.759989:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82085880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022119424:0@lo:400:kworker.0 00000100:00100000:2.0:1789762608.760009:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82084700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022119040:0@lo:400:kworker.0 00000100:00100000:2.0:1789762608.760065:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022118912 00000100:00100000:2.0:1789762608.760070:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022118912 00000100:00100000:2.0:1789762608.760149:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 54 00000100:00100000:2.0:1789762608.760162:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022118912:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762608.760220:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022118912:12345-0@lo:400:kworker.0 Request processed in 61us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762608.760229:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 54 00000100:00100000:0.0:1789762608.760269:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82087480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022118912:0@lo:400:kworker.0 00000001:00080000:0.0:1789762608.875113:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967405 is committed 00000100:00100000:0.0:1789762608.878401:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762609.075500:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967406 is committed 00000100:00100000:0.0:1789762609.078627:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762609.079199:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762609.276004:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967407 is committed 00000001:00080000:0.0:1789762609.475325:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967408 is committed 00000100:00100000:2.0:1789762609.478460:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014787200 00000100:00100000:2.0:1789762609.478468:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014787200 00000100:00100000:2.0:1789762609.478505:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 132767337976 time 132767394306 state 0x0 ref 1 00000100:00100000:2.0:1789762609.478521:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 132967394306 time 132767394306 state 0x1 ref 1 00000001:00080000:0.0:1789762609.675665:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967409 is committed 00000001:00080000:2.0:1789762609.875341:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967410 is committed 00000100:00100000:0.0:1789762609.878051:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762610.063741:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 133367540949 time 133367643726 state 0x0 ref 1 00000100:00100000:2.0:1789762610.063777:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014787456:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762610.063849:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 88080384, mtime: 0x6aad9c31, atime: 0x6aad9c21, ctime: 0x6aad9c31, blocks: 0x2a009 00010000:00010000:2.0:1789762610.063858:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b01906 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762610.063871:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=133s v=(0 0) 00010000:00010000:2.0:1789762610.063875:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762610.063880:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b01906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762610.063891:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008cb18182/0x416193e012b01906 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762610.074886:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967411 is committed 00010000:00010000:2.0:1789762610.103233:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762610.275473:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967412 is committed 00002000:00010000:2.0:1789762610.275510:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb size from disk: 89128960 -> 90177536 00002000:00010000:2.0:1789762610.275513:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x5:0x0] updating lvb blocks from disk: 174089 -> 176137 00010000:00010000:2.0:1789762610.275523:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b0ea649/0x416193e012b0190d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762610.275533:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b0ea649/0x416193e012b0190d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762610.275545:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b0ea649/0x416193e012b0190d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762610.275551:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa64000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762610.275554:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b0ea649/0x416193e012b0190d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762610.275561:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b0ea649/0x416193e012b0190d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762610.275575:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b0ea649/0x416193e012b0190d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762610.275581:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b0ea649/0x416193e012b0190d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762610.275642:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014787584:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11772us (198298us total) trans 4294967412 rc 0/0 00000100:00100000:2.0:1789762610.275650:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 120 00000100:00100000:2.0:1789762610.275656:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 133567643726 time 133567753098 state 0x2 ref 0 00000100:00100000:0.0:1789762610.278029:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762610.477245:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967413 is committed 00000100:00100000:2.0:1789762610.480346:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762610.675954:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967414 is committed 00000100:00100000:2.0:1789762610.679064:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762610.876496:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967415 is committed 00000100:00100000:0.0:1789762610.879522:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762611.076371:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967416 is committed 00000100:00100000:2.0:1789762611.078900:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762611.127196:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762611.276885:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967417 is committed 00000100:00100000:0.0:1789762611.280002:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762611.476295:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967418 is committed 00000100:00100000:0.0:1789762611.478946:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762611.664358:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 134968170642 time 134968261194 state 0x0 ref 1 00000100:00100000:2.0:1789762611.664385:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf512d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014788480:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762611.664433:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 96468992, mtime: 0x6aad9c33, atime: 0x6aad9c21, ctime: 0x6aad9c33, blocks: 0x2e00b 00010000:00010000:2.0:1789762611.664447:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0193e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762611.664464:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=134s v=(0 0) 00010000:00010000:2.0:1789762611.664467:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762611.664472:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0193e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762611.664483:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001dde7086/0x416193e012b0193e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762611.676297:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967419 is committed 00000100:00100000:0.0:1789762611.678755:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762611.876153:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967420 is committed 00000100:00100000:2.0:1789762611.878724:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014788736 00000100:00100000:2.0:1789762611.878733:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014788736 00000100:00100000:2.0:1789762611.878789:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 135168261194 time 135168338112 state 0x0 ref 1 00000100:00100000:2.0:1789762611.878799:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 135368338112 time 135168338112 state 0x1 ref 1 00000001:00080000:0.0:1789762612.077839:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967421 is committed 00010000:00010000:0.0:1789762612.151216:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762612.276244:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967422 is committed 00000100:00100000:0.0:1789762612.279249:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762612.464684:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 135768512187 time 135768586591 state 0x0 ref 1 00000100:00100000:2.0:1789762612.464710:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf512a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014788992:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762612.464781:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 100663296, mtime: 0x6aad9c34, atime: 0x6aad9c21, ctime: 0x6aad9c34, blocks: 0x3000b 00010000:00010000:2.0:1789762612.464802:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b0195a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762612.464816:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=135s v=(0 0) 00010000:00010000:2.0:1789762612.464820:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762612.464824:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b0195a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762612.464839:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d40a30f1/0x416193e012b0195a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762612.476901:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967423 is committed 00000100:00100000:0.0:1789762612.481210:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014789120 00000100:00100000:0.0:1789762612.481221:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014789120 00000100:00100000:0.0:1789762612.481269:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 135768512187 time 135768586591 state 0x0 ref 1 00000100:00100000:0.0:1789762612.481279:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 135968586591 time 135768586591 state 0x1 ref 1 00000001:00080000:2.0:1789762612.676551:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967424 is committed 00000100:00100000:0.0:1789762612.679665:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762612.877724:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967425 is committed 00000100:00100000:0.0:1789762612.880243:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762613.064953:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 136368771761 time 136368855362 state 0x0 ref 1 00000100:00100000:0.0:1789762613.064980:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf511180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014789376:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762613.065031:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 103809024, mtime: 0x6aad9c34, atime: 0x6aad9c21, ctime: 0x6aad9c34, blocks: 0x3180b 00010000:00010000:0.0:1789762613.065049:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b0196f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762613.065062:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=136s v=(0 0) 00010000:00010000:0.0:1789762613.065066:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762613.065071:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b0196f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762613.065117:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b0196f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762613.076543:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967426 is committed 00000100:00100000:0.0:1789762613.128324:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762613.128386:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014789888 00000100:00100000:2.0:1789762613.128396:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014789888 00000100:00100000:2.0:1789762613.128451:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 82 00000100:00100000:2.0:1789762613.128459:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf513100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:10526:x1876702014789888:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:2.0:1789762613.128487:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762613.128538:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01976 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd0ac expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.128611:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0197d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.128623:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0197d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.128632:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0197d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.128659:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1c00/0x416193e012b01984 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.128664:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1c00/0x416193e012b01984 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.128669:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1c00/0x416193e012b01984 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129116:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0000/0x416193e012b0198b lrc: 3/1,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129124:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0000/0x416193e012b0198b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129128:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0000/0x416193e012b0198b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762613.129136:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xb:0x0]: returning size 104857600 00010000:00010000:2.0:1789762613.129206:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0197d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129212:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0197d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129217:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0197d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129221:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e1600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762613.129224:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0197d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129237:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0197d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129246:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:2.0:1789762613.129249:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762613.129252:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0197d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129256:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1600/0x416193e012b0197d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129265:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1c00/0x416193e012b01984 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129269:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1c00/0x416193e012b01984 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129272:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1c00/0x416193e012b01984 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129276:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e1c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762613.129278:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1c00/0x416193e012b01984 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129282:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1c00/0x416193e012b01984 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129286:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x9:0x0].0x4009f790 (ffff981e82a26d00) 00010000:00010000:2.0:1789762613.129287:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762613.129289:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1c00/0x416193e012b01984 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129298:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e1c00/0x416193e012b01984 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x9:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129336:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0c00/0x416193e012b01976 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x9:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0ac expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129352:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebc2e0000/0x416193e012b0198b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0ac expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.129361:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff981e87ddf500) 00010000:00010000:2.0:1789762613.129364:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762613.129366:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebc2e0000, rc 0) 00000100:00100000:2.0:1789762613.129415:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf513100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:10526:x1876702014789888:12345-192.168.203.48@tcp:101:dd.500 Request processed in 957us (1099us total) trans 4294967351 rc 0/0 00000100:00100000:2.0:1789762613.129424:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 82 00000100:00100000:0.0:1789762613.130799:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762613.175210:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762613.265055:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 136568855362 time 136568957671 state 0x0 ref 1 00000100:00100000:2.0:1789762613.265081:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014790144:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762613.265158:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b019a0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.265173:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762613.265176:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762613.265181:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b019a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.265207:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b019a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.269337:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b019a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.269348:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b019a0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.269355:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b019a0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.269361:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762613.269365:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b019a0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.269371:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b019a0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.269385:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b019a0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762613.269391:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c25dd99/0x416193e012b019a0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762613.269438:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014790144:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4368us (135988us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762613.269451:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 135 00000100:00100000:2.0:1789762613.269457:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 136568855362 time 136568957671 state 0x2 ref 0 00000100:00100000:0.0:1789762613.346313:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762613.351417:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014790528 00000100:00100000:0.0:1789762613.351426:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014790528 00000100:00100000:0.0:1789762613.351482:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 55 00000100:00100000:0.0:1789762613.351493:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2362:x1876702014790528:12345-192.168.203.48@tcp:9:kworker.0 00000020:00000080:0.0:1789762613.351529:0:6649:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x416193e012b0136b 00000100:00080000:0.0:1789762613.351537:0:6649:0:(import.c:65:import_set_state_nolock()) ffff981eb48bf800 : changing import state from FULL to CLOSED 00000001:00000004:0.0:1789762613.351569:0:6649:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967321 00010000:00010000:2.0:1789762613.351569:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762613.351816:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+2:2362:x1876702014790528:12345-192.168.203.48@tcp:9:kworker.0 Request processed in 325us (459us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762613.351826:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 55 00000100:00100000:0.0:1789762613.367301:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022119552 00000100:00100000:0.0:1789762613.367307:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022119552 00000100:00100000:0.0:1789762613.367357:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 27 00000100:00100000:0.0:1789762613.367369:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022119552:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762613.367388:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3762290688 avail 3752853504 00000100:00100000:0.0:1789762613.367439:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022119552:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (174us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762613.367445:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 27 00000100:00100000:2.0:1789762613.471445:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762613.672322:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762613.870960:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762613.879231:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82085f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022119680:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762613.879239:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead4f4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022119936:0@lo:400:kworker.0 00000100:00100000:0.0:1789762613.879295:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762613.879302:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762613.879366:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead4f5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022120192:0@lo:400:kworker.0 00000100:00100000:2.0:1789762613.879367:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d84543b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022120448:0@lo:400:kworker.0 00000100:00100000:0.0:1789762613.879406:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762613.879428:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022120064 00000100:00100000:2.0:1789762613.879434:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022120064 00000100:00100000:0.0:1789762613.879456:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022119680 00000100:00100000:0.0:1789762613.879462:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022119680 00000100:00100000:2.0:1789762613.879482:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 56 00000100:00100000:2.0:1789762613.879495:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022119936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762613.879514:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 28 00000100:00100000:0.0:1789762613.879529:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf511500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022119680:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762613.879553:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022119936:12345-0@lo:400:kworker.0 Request processed in 62us (250us total) trans 0 rc 0/0 00002000:00000024:0.0:1789762613.879557:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762613.879568:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 56 00000100:00100000:2.0:1789762613.879589:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 58 00000100:00100000:2.0:1789762613.879599:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022120320:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762613.879609:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf511500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022119680:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (314us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762613.879618:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 28 00000100:00100000:2.0:1789762613.879645:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022120320:12345-0@lo:400:kworker.0 Request processed in 48us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762613.879647:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022120192 00000100:00100000:0.0:1789762613.879653:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022120192 00000100:00100000:2.0:1789762613.879663:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 58 00000100:00100000:0.0:1789762613.879684:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 34 00000100:00080000:2.0:1789762613.879693:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762613.879695:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022120192:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762613.879697:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762613.879725:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762613.879735:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022120192:12345-0@lo:400:kworker.0 Request processed in 44us (330us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762613.879767:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 34 00000100:00100000:2.0:1789762613.879777:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022119936:0@lo:400:kworker.0 00000100:00100000:0.0:1789762613.879786:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022120448 00000100:00100000:0.0:1789762613.879790:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022120448 00000100:00100000:0.0:1789762613.879821:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 35 00000100:00100000:0.0:1789762613.879829:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82086d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022120448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762613.879867:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82086d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022120448:12345-0@lo:400:kworker.0 Request processed in 40us (145us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762613.879873:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 35 00000100:00100000:0.0:1789762613.879910:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022120192:0@lo:400:kworker.0 00000100:00100000:0.0:1789762613.879928:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82085f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022119680:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762614.065389:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 137369210121 time 137369290311 state 0x0 ref 1 00000100:00100000:2.0:1789762614.065448:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf510000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014791040:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762614.065486:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000026893f01/0x416193e012b019bc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762614.065500:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762614.065503:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762614.065508:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000026893f01/0x416193e012b019bc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762614.065520:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000026893f01/0x416193e012b019bc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762614.069018:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000026893f01/0x416193e012b019bc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762614.069031:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000026893f01/0x416193e012b019bc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762614.069038:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000026893f01/0x416193e012b019bc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762614.069045:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec04e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762614.069049:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000026893f01/0x416193e012b019bc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762614.069055:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000026893f01/0x416193e012b019bc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762614.069077:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000026893f01/0x416193e012b019bc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762614.069082:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000026893f01/0x416193e012b019bc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762614.069148:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf510000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014791040:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3725us (198188us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762614.069155:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 139 00000100:00100000:2.0:1789762614.069161:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 137369210121 time 137369290311 state 0x2 ref 0 00000100:00100000:2.0:1789762614.071231:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762614.199211:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762614.271603:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762614.471810:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762614.672129:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762614.871533:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762615.073480:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762615.223218:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:2.0:1789762615.423162:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:2.0:1789762615.423170:0:7605:0:(genops.c:885:class_export_put()) final put ffff981eb48bb000/f1bd28de-ee78-4d18-bad3-fa7cec968893 00000100:00100000:0.0:1789762615.666085:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 138969823647 time 138969965212 state 0x0 ref 1 00000100:00100000:0.0:1789762615.666172:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014792064:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762615.666237:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b019f4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762615.666259:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762615.666265:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762615.666272:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b019f4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762615.666316:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b019f4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762615.672338:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b019f4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762615.672357:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b019f4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762615.672370:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b019f4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762615.672380:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e88522a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762615.672387:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b019f4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762615.672399:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b019f4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762615.672425:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b019f4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762615.672435:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b019f4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762615.672531:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014792064:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6395us (200972us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762615.672545:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 147 00000100:00100000:0.0:1789762615.672554:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 138969823647 time 138969965212 state 0x2 ref 0 00000100:00100000:0.0:1789762615.675662:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762615.874117:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762616.247254:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762616.269980:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01a09 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762616.269996:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01a09 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762616.270003:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01a09 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762616.270028:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762616.270032:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01a09 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762616.270039:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01a09 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762616.270059:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01a09 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762616.270064:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cc442cac/0x416193e012b01a09 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762616.270199:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014792448:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3771us (196112us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762616.270210:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 150 00000100:00100000:2.0:1789762616.270220:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 139570188286 time 139570312979 state 0x2 ref 0 00000100:00100000:2.0:1789762616.272519:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762616.466515:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 139770312979 time 139770417394 state 0x0 ref 1 00000100:00100000:0.0:1789762616.466542:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbe0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014792576:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762616.466579:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008fcaad3a/0x416193e012b01a10 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762616.466593:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762616.466596:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762616.466601:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008fcaad3a/0x416193e012b01a10 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762616.466612:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008fcaad3a/0x416193e012b01a10 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762616.471792:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762616.471854:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014792704 00000100:00100000:2.0:1789762616.471868:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014792704 00000100:00100000:2.0:1789762616.471923:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 139770312979 time 139770417394 state 0x0 ref 1 00000100:00100000:2.0:1789762616.471934:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 139970417394 time 139770417394 state 0x1 ref 1 00000100:00100000:2.0:1789762616.871631:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762617.066785:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 140370575614 time 140370686788 state 0x0 ref 1 00000100:00100000:2.0:1789762617.066814:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbe0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014792960:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762617.066854:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bfbd47c7/0x416193e012b01a25 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.066872:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762617.066877:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762617.066899:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bfbd47c7/0x416193e012b01a25 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.066914:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bfbd47c7/0x416193e012b01a25 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.070650:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bfbd47c7/0x416193e012b01a25 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.070662:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bfbd47c7/0x416193e012b01a25 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.070669:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bfbd47c7/0x416193e012b01a25 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.070675:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e902da000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762617.070679:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bfbd47c7/0x416193e012b01a25 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.070698:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bfbd47c7/0x416193e012b01a25 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.070713:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bfbd47c7/0x416193e012b01a25 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.070718:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bfbd47c7/0x416193e012b01a25 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762617.070780:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbe0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014792960:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3977us (199148us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762617.070788:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 154 00000100:00100000:2.0:1789762617.070794:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 140370575614 time 140370686788 state 0x2 ref 0 00000100:00100000:2.0:1789762617.072773:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762617.271184:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762617.272460:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014793216 00000100:00100000:2.0:1789762617.272468:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014793216 00000100:00100000:2.0:1789762617.272499:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 140570686788 time 140570731421 state 0x0 ref 1 00000100:00100000:2.0:1789762617.272508:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 140770731421 time 140570731421 state 0x1 ref 1 00000100:00100000:2.0:1789762617.867030:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141170858664 time 141170933063 state 0x0 ref 1 00000100:00100000:2.0:1789762617.867057:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014793472:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762617.867113:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a41 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.867140:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762617.867143:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762617.867148:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a41 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.867161:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a41 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.870109:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a41 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.870119:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a41 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.870126:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a41 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.870130:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762617.870134:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a41 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.870140:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a41 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.870152:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a41 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762617.870168:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a41 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762617.870215:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014793472:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3171us (197864us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762617.870221:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 158 00000100:00100000:2.0:1789762617.870226:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141170858664 time 141170933063 state 0x2 ref 0 00000100:00100000:0.0:1789762617.872410:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762618.067204:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141370933063 time 141371106383 state 0x0 ref 1 00000100:00100000:0.0:1789762618.067242:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf513b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014793600:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762618.067299:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d3336d3d/0x416193e012b01a48 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.067315:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762618.067319:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762618.067323:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3336d3d/0x416193e012b01a48 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.067335:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3336d3d/0x416193e012b01a48 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.070836:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d3336d3d/0x416193e012b01a48 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.070850:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3336d3d/0x416193e012b01a48 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.070861:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3336d3d/0x416193e012b01a48 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.070869:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac65a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762618.070895:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3336d3d/0x416193e012b01a48 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.070915:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3336d3d/0x416193e012b01a48 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.070970:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3336d3d/0x416193e012b01a48 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.070981:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3336d3d/0x416193e012b01a48 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762618.071065:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf513b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014793600:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3841us (198650us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762618.071083:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 159 00000100:00100000:0.0:1789762618.071155:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141370933063 time 141371106383 state 0x2 ref 0 00000100:00100000:0.0:1789762618.073586:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762618.073657:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014793728 00000100:00100000:2.0:1789762618.073666:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014793728 00000100:00100000:2.0:1789762618.073703:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141370933063 time 141371106383 state 0x0 ref 1 00000100:00100000:2.0:1789762618.073713:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141571106383 time 141371106383 state 0x1 ref 1 00000100:00100000:0.0:1789762618.272694:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014793856 00000100:00100000:0.0:1789762618.272702:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014793856 00000100:00100000:0.0:1789762618.272747:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141571106383 time 141571204779 state 0x0 ref 1 00000100:00100000:0.0:1789762618.272756:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141771204779 time 141571204779 state 0x1 ref 1 00010000:00010000:0.0:1789762618.295181:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762618.466373:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762618.466412:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762618.466463:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762618.466477:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014794112 00000100:00100000:2.0:1789762618.466485:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014794112 00000100:00080000:0.0:1789762618.466542:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762618.466558:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 59 00000100:00100000:2.0:1789762618.466569:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702014794112:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762618.466639:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702014794112:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 64us (220us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762618.466648:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 59 00010000:00010000:0.0:1789762618.470216:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b404de1b/0x416193e012b01a56 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.470228:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b404de1b/0x416193e012b01a56 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.470241:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b404de1b/0x416193e012b01a56 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.470247:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa65e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762618.470251:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b404de1b/0x416193e012b01a56 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.470258:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b404de1b/0x416193e012b01a56 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.470273:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b404de1b/0x416193e012b01a56 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.470279:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b404de1b/0x416193e012b01a56 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762618.470317:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014793856:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2948us (197663us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762618.470324:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 161 00000100:00100000:0.0:1789762618.470330:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141771204779 time 141771265973 state 0x2 ref 0 00000100:00100000:0.0:1789762618.472281:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762618.487202:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762618.487218:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d84542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022120576:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762618.487258:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762618.487297:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022120576 00000100:00100000:0.0:1789762618.487303:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022120576 00000100:00080000:2.0:1789762618.487336:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762618.487360:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 29 00000100:00100000:0.0:1789762618.487373:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbe2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022120576:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762618.487408:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762618.487466:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbe2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022120576:12345-0@lo:13:osp-pre-0-0.0 Request processed in 96us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762618.487475:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 29 00000100:00100000:2.0:1789762618.487505:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d84542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022120576:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762618.667472:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141971265973 time 141971375096 state 0x0 ref 1 00000100:00100000:2.0:1789762618.667498:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014794368:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762618.667535:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a5d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762618.667548:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762618.667552:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762618.667557:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a5d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762618.667567:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a5d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762618.670596:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a5d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762618.670606:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a5d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762618.670612:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a5d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762618.670617:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762618.670620:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a5d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762618.670638:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a5d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762618.670651:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a5d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762618.670656:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b01a5d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762618.670704:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014794368:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3217us (198424us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762618.670711:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 162 00000100:00100000:2.0:1789762618.670736:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141971265973 time 141971375096 state 0x2 ref 0 00000100:00100000:0.0:1789762618.672470:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014794496 00000100:00100000:0.0:1789762618.672478:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014794496 00000100:00100000:0.0:1789762618.672509:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 141971265973 time 141971375096 state 0x0 ref 1 00000100:00100000:0.0:1789762618.672517:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 142171375096 time 141971375096 state 0x1 ref 1 00000100:00100000:0.0:1789762618.867528:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 142171375096 time 142171430631 state 0x0 ref 1 00000100:00100000:0.0:1789762618.867555:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014794496:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762618.867593:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001e5cd90d/0x416193e012b01a64 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.867607:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762618.867610:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762618.867615:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001e5cd90d/0x416193e012b01a64 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762618.867627:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001e5cd90d/0x416193e012b01a64 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762618.872890:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014794624 00000100:00100000:2.0:1789762618.872898:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014794624 00000100:00100000:2.0:1789762618.872929:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 142171375096 time 142171430631 state 0x0 ref 1 00000100:00100000:2.0:1789762618.872937:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 142371430631 time 142171430631 state 0x1 ref 1 00000100:00100000:0.0:1789762618.999325:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfbe0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022120960:0@lo:400:kworker.0 00000100:00100000:2.0:1789762618.999366:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfbe1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022121088:0@lo:400:kworker.0 00000100:00100000:0.0:1789762618.999401:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762618.999440:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022120832 00000100:00100000:2.0:1789762618.999447:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022120832 00000100:00100000:0.0:1789762618.999458:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfbe0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022121472:0@lo:400:kworker.0 00000100:00100000:2.0:1789762618.999507:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30 00000100:00100000:2.0:1789762618.999521:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022120832:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762618.999522:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:2.0:1789762618.999555:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762618.999584:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022121216 00000100:00100000:0.0:1789762618.999592:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022121216 00000100:00100000:2.0:1789762618.999622:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022120832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 105us (227us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762618.999630:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30 00000100:00100000:0.0:1789762618.999641:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022121472 00000100:00100000:0.0:1789762618.999647:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022121472 00000100:00100000:2.0:1789762618.999687:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 36 00000100:00100000:0.0:1789762618.999699:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 37 00000100:00100000:2.0:1789762618.999699:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84543800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022121216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762618.999737:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022121472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762618.999778:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022121216:12345-0@lo:400:kworker.0 Request processed in 82us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762618.999790:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 36 00000100:00100000:2.0:1789762618.999834:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022121088 00000100:00100000:2.0:1789762618.999841:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022121088 00000100:00100000:0.0:1789762618.999854:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022121472:12345-0@lo:400:kworker.0 Request processed in 137us (333us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762618.999873:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 37 00000100:00100000:2.0:1789762618.999890:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 62 00000100:00100000:2.0:1789762618.999906:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022121344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762618.999944:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 63 00000100:00100000:0.0:1789762618.999969:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022121088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762618.999985:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022121344:12345-0@lo:400:kworker.0 Request processed in 82us (482us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762618.999996:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 62 00000100:00100000:0.0:1789762619.000067:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022121088:12345-0@lo:400:kworker.0 Request processed in 103us (497us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762619.000087:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfbe1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022121088:0@lo:400:kworker.0 00000100:00100000:0.0:1789762619.000088:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 63 00000100:00100000:2.0:1789762619.072668:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762619.267782:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 142571532028 time 142571663398 state 0x0 ref 1 00000100:00100000:0.0:1789762619.267839:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbe1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014794752:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762619.267884:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004a2c627d/0x416193e012b01a72 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762619.267922:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762619.267928:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762619.267936:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a2c627d/0x416193e012b01a72 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762619.267953:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a2c627d/0x416193e012b01a72 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762619.274417:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762619.319241:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762619.467844:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 142771663398 time 142771746226 state 0x0 ref 1 00000100:00100000:2.0:1789762619.467871:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbe3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014794880:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762619.467907:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bc9d6518/0x416193e012b01a79 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762619.467921:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762619.467925:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762619.467943:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bc9d6518/0x416193e012b01a79 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762619.467955:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bc9d6518/0x416193e012b01a79 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762619.474034:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762619.667945:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 142971746226 time 142971843179 state 0x0 ref 1 00000100:00100000:0.0:1789762619.667983:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014795008:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762619.668047:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a80 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762619.668065:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762619.668069:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762619.668074:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a80 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762619.668120:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a80 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762619.672542:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a80 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762619.672555:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a80 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762619.672563:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a80 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762619.672590:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762619.672596:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a80 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762619.672604:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a80 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762619.672624:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a80 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762619.672631:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a80 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762619.672699:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014795008:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4735us (198671us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762619.672707:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 167 00000100:00100000:0.0:1789762619.672726:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 142971746226 time 142971843179 state 0x2 ref 0 00000100:00100000:0.0:1789762619.675487:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762619.873941:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762619.874003:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014795264 00000100:00100000:0.0:1789762619.874014:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014795264 00000100:00100000:0.0:1789762619.874065:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 143171843179 time 143171941086 state 0x0 ref 1 00000100:00100000:0.0:1789762619.874076:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 143371941086 time 143171941086 state 0x1 ref 1 00000100:00100000:0.0:1789762620.068149:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 143371941086 time 143372046864 state 0x0 ref 1 00000100:00100000:0.0:1789762620.068227:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbe2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014795264:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762620.068285:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a8e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.068303:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762620.068309:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762620.068315:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a8e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.068330:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a8e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.072145:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a8e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.072167:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a8e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.072179:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a8e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.072189:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762620.072195:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a8e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.072208:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a8e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.072236:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a8e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.072243:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a8e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762620.072301:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbe2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014795264:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4107us (198369us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762620.072309:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 169 00000100:00100000:0.0:1789762620.072315:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 143371941086 time 143372046864 state 0x2 ref 0 00010000:00010000:0.0:1789762620.271820:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002f2b3d1d/0x416193e012b01a95 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.271834:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f2b3d1d/0x416193e012b01a95 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.271841:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f2b3d1d/0x416193e012b01a95 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.271846:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ead6e1400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762620.271850:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f2b3d1d/0x416193e012b01a95 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.271857:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f2b3d1d/0x416193e012b01a95 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.271888:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f2b3d1d/0x416193e012b01a95 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.271894:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f2b3d1d/0x416193e012b01a95 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762620.271956:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014795392:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3646us (197578us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762620.271965:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 170 00000100:00100000:0.0:1789762620.271971:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 143572046864 time 143572188118 state 0x2 ref 0 00000100:00100000:2.0:1789762620.274137:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762620.274180:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014795520 00000100:00100000:0.0:1789762620.274193:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014795520 00000100:00100000:0.0:1789762620.274237:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 143572046864 time 143572188118 state 0x0 ref 1 00000100:00100000:0.0:1789762620.274249:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 143772188118 time 143572188118 state 0x1 ref 1 00010000:00010000:2.0:1789762620.343273:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762620.468389:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 143772188118 time 143772283406 state 0x0 ref 1 00000100:00100000:0.0:1789762620.468432:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbe3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014795520:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762620.468484:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a9c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.468503:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762620.468507:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762620.468513:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a9c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762620.468555:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b01a9c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762620.474441:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762620.474494:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014795648 00000100:00100000:0.0:1789762620.474504:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014795648 00000100:00100000:0.0:1789762620.474548:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 143772188118 time 143772283406 state 0x0 ref 1 00000100:00100000:0.0:1789762620.474564:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 143972283406 time 143772283406 state 0x1 ref 1 00000100:00100000:0.0:1789762621.068771:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 144372464589 time 144372665893 state 0x0 ref 1 00000100:00100000:0.0:1789762621.068822:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014795904:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762621.068890:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.068913:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762621.068918:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762621.068926:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.068951:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.072609:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.072654:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.072665:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.072674:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e902daa00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762621.072680:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.072692:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.072735:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.072745:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762621.072837:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014795904:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4036us (198510us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762621.072853:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 174 00000100:00100000:0.0:1789762621.072863:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 144372464589 time 144372665893 state 0x2 ref 0 00000100:00100000:0.0:1789762621.075023:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014796032 00000100:00100000:0.0:1789762621.075061:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014796032 00000100:00100000:0.0:1789762621.075147:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 144372464589 time 144372665893 state 0x0 ref 1 00000100:00100000:0.0:1789762621.075160:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 144572665893 time 144372665893 state 0x1 ref 1 00000100:00100000:0.0:1789762621.268926:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 144572665893 time 144572824719 state 0x0 ref 1 00000100:00100000:0.0:1789762621.268964:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014796032:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762621.269022:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.269042:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762621.269047:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762621.269054:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.269070:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.273061:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.273077:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.273126:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.273162:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e902daa00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762621.273170:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.273184:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.273203:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.273212:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070dcc23f/0x416193e012b01ab8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762621.273297:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014796032:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4349us (198362us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762621.273310:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 175 00000100:00100000:0.0:1789762621.273317:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 144572665893 time 144572824719 state 0x2 ref 0 00000100:00100000:0.0:1789762621.275685:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014796160 00000100:00100000:0.0:1789762621.275727:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014796160 00000100:00100000:0.0:1789762621.275789:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 144572665893 time 144572824719 state 0x0 ref 1 00000100:00100000:0.0:1789762621.275802:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 144772824719 time 144572824719 state 0x1 ref 1 00010000:00010000:2.0:1789762621.367308:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762621.475907:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762621.475987:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014796288 00000100:00100000:0.0:1789762621.475997:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014796288 00000100:00100000:0.0:1789762621.476063:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 144772824719 time 144772963931 state 0x0 ref 1 00000100:00100000:0.0:1789762621.476085:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 144972963931 time 144772963931 state 0x1 ref 1 00000100:00100000:0.0:1789762621.677115:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762621.869328:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 145173060466 time 145173224923 state 0x0 ref 1 00000100:00100000:0.0:1789762621.869370:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014796416:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762621.869428:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a68de151/0x416193e012b01acd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.869451:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762621.869457:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762621.869465:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a68de151/0x416193e012b01acd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.869487:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a68de151/0x416193e012b01acd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.873996:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a68de151/0x416193e012b01acd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.874017:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a68de151/0x416193e012b01acd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.874030:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a68de151/0x416193e012b01acd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.874040:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebff18400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762621.874066:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a68de151/0x416193e012b01acd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.874078:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a68de151/0x416193e012b01acd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.874133:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a68de151/0x416193e012b01acd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762621.874144:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a68de151/0x416193e012b01acd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762621.874229:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014796416:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4877us (197145us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762621.874242:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 178 00000100:00100000:0.0:1789762621.874252:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 145173060466 time 145173224923 state 0x2 ref 0 00000100:00100000:0.0:1789762621.876388:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:0.0:1789762622.069537:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762622.078328:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762622.078403:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014796672 00000100:00100000:0.0:1789762622.078415:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014796672 00000100:00100000:0.0:1789762622.078459:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 145373224923 time 145373337943 state 0x0 ref 1 00000100:00100000:0.0:1789762622.078472:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 145573337943 time 145373337943 state 0x1 ref 1 00000100:00100000:0.0:1789762622.277304:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762622.391348:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762622.475372:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b01ae2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.475428:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b01ae2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.475449:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b01ae2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.475464:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec05c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762622.475472:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b01ae2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.475486:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b01ae2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.475516:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b01ae2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.475528:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b01ae2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762622.475611:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014796800:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5853us (198313us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762622.475625:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 181 00000100:00100000:0.0:1789762622.475635:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 145773488998 time 145773623719 state 0x2 ref 0 00000100:00100000:2.0:1789762622.479560:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762622.669894:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 145973623719 time 145973791579 state 0x0 ref 1 00000100:00100000:0.0:1789762622.669956:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbe3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014796928:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762622.670028:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01ae9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.670052:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762622.670058:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762622.670066:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01ae9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.670129:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01ae9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.674506:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01ae9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.674537:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01ae9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.674546:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01ae9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.674553:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762622.674557:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01ae9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.674565:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01ae9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.674591:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01ae9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762622.674598:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01ae9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762622.674660:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbe3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014796928:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4719us (195103us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762622.674674:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 182 00000100:00100000:0.0:1789762622.674683:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 145973623719 time 145973791579 state 0x2 ref 0 00000100:00100000:0.0:1789762622.677649:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762622.677705:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014797056 00000100:00100000:2.0:1789762622.677714:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014797056 00000100:00100000:2.0:1789762622.677777:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 145973623719 time 145973791579 state 0x0 ref 1 00000100:00100000:2.0:1789762622.677793:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 146173791579 time 145973791579 state 0x1 ref 1 00000100:00100000:0.0:1789762622.875597:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014797184 00000100:00100000:0.0:1789762622.875605:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014797184 00000100:00100000:0.0:1789762622.875637:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 146173791579 time 146173911238 state 0x0 ref 1 00000100:00100000:0.0:1789762622.875646:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 146373911238 time 146173911238 state 0x1 ref 1 00000100:00100000:2.0:1789762623.076746:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762623.270207:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 146574019738 time 146574108188 state 0x0 ref 1 00000100:00100000:2.0:1789762623.270240:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbe0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014797312:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762623.270279:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b01afe lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762623.270294:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762623.270297:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762623.270316:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b01afe lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762623.270328:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b01afe lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762623.275627:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762623.415190:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762623.470338:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 146774108188 time 146774237755 state 0x0 ref 1 00000100:00100000:0.0:1789762623.470372:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbe0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014797440:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762623.470417:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b01b05 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762623.470434:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762623.470463:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762623.470469:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b01b05 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762623.470480:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b01b05 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762623.476134:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762623.586416:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762623.586456:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762623.586502:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762623.586544:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762623.586568:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 64 00000100:00100000:2.0:1789762623.586584:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 65 00000100:00100000:0.0:1789762623.586589:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702014797824:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762623.586602:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014797952:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762623.586675:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014797952:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 78us (173us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762623.586694:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702014797824:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 98us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762623.586705:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 65 00000100:00100000:0.0:1789762623.586713:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 64 00000100:00100000:2.0:1789762623.876083:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:0.0:1789762624.070613:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762624.076053:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762624.119229:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebf47e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022121856:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762624.119300:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb14700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022121984:0@lo:400:kworker.0 00000100:00100000:0.0:1789762624.119315:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762624.119347:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762624.119372:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb14380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022122496:0@lo:400:kworker.0 00000100:00100000:0.0:1789762624.119382:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb15880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022122368:0@lo:400:kworker.0 00000100:00100000:2.0:1789762624.119406:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762624.119421:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762624.119438:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762624.119449:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb15c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022122112:0@lo:400:kworker.0 00000100:00100000:0.0:1789762624.119464:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022121984 00000100:00100000:0.0:1789762624.119470:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022121984 00000100:00100000:2.0:1789762624.119482:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762624.119514:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022122368 00000100:00100000:0.0:1789762624.119521:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022122368 00000100:00100000:2.0:1789762624.119523:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022122112 00000100:00100000:2.0:1789762624.119527:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022122112 00000100:00100000:2.0:1789762624.119577:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 66 00000100:00100000:0.0:1789762624.119596:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 67 00000100:00100000:0.0:1789762624.119606:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022122368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762624.119627:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022121984:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762624.119674:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022122368:12345-0@lo:400:kworker.0 Request processed in 68us (250us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762624.119674:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022121984:12345-0@lo:400:kworker.0 Request processed in 77us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762624.119698:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 66 00000100:00100000:0.0:1789762624.119705:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 67 00000100:00100000:0.0:1789762624.119740:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 39 00000100:00100000:0.0:1789762624.119756:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb494c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022122240:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762624.119756:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb14700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022121984:0@lo:400:kworker.0 00000100:00100000:2.0:1789762624.119779:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb14380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022122496:0@lo:400:kworker.0 00000100:00100000:0.0:1789762624.119808:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb494c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022122240:12345-0@lo:400:kworker.0 Request processed in 58us (372us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762624.119821:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 39 00000100:00100000:2.0:1789762624.119821:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb15c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022122112:0@lo:400:kworker.0 00000100:00100000:2.0:1789762624.119869:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb15f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022122240:0@lo:400:kworker.0 00000100:00100000:2.0:1789762624.119885:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfbe1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022121728:0@lo:400:kworker.0 00000100:00080000:2.0:1789762624.119931:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762624.119938:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762624.119974:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb15880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022122368:0@lo:400:kworker.0 00000100:00100000:2.0:1789762624.119993:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebf47e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022121856:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762624.275720:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762624.439259:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762624.476156:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762624.676471:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762624.876391:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762625.076497:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762625.276460:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762625.463272:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762625.476247:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762625.677642:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762625.877542:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762626.071310:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762626.075761:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007f865058/0x416193e012b01b60 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.075774:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f865058/0x416193e012b01b60 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.075783:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f865058/0x416193e012b01b60 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.075790:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812dc00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762626.075793:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f865058/0x416193e012b01b60 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.075801:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f865058/0x416193e012b01b60 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.075832:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f865058/0x416193e012b01b60 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.075839:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f865058/0x416193e012b01b60 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762626.075912:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014799488:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4647us (198353us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762626.075921:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 199 00000100:00100000:0.0:1789762626.075929:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 149375056311 time 149375130619 state 0x2 ref 0 00000100:00100000:2.0:1789762626.078929:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762626.277834:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014799744 00000100:00100000:0.0:1789762626.277843:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014799744 00000100:00100000:0.0:1789762626.277881:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 149575130619 time 149575189393 state 0x0 ref 1 00000100:00100000:0.0:1789762626.277893:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 149775189393 time 149575189393 state 0x1 ref 1 00010000:00010000:0.0:1789762626.487196:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762626.677888:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762626.871626:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 150175448514 time 150175527699 state 0x0 ref 1 00000100:00100000:0.0:1789762626.871665:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014800000:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762626.871701:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005b27641f/0x416193e012b01b7c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.871715:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762626.871718:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762626.871735:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b27641f/0x416193e012b01b7c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.871750:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b27641f/0x416193e012b01b7c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.875029:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005b27641f/0x416193e012b01b7c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.875038:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005b27641f/0x416193e012b01b7c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.875045:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005b27641f/0x416193e012b01b7c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.875051:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4e2200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762626.875055:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005b27641f/0x416193e012b01b7c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.875061:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005b27641f/0x416193e012b01b7c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.875076:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005b27641f/0x416193e012b01b7c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762626.875081:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b27641f/0x416193e012b01b7c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762626.875169:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014800000:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3527us (197281us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762626.875178:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 203 00000100:00100000:0.0:1789762626.875183:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 150175448514 time 150175527699 state 0x2 ref 0 00000100:00100000:0.0:1789762626.877142:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762627.076794:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014800256 00000100:00100000:0.0:1789762627.076803:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014800256 00000100:00100000:0.0:1789762627.076836:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 150375527699 time 150375591378 state 0x0 ref 1 00000100:00100000:0.0:1789762627.076844:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 150575591378 time 150375591378 state 0x1 ref 1 00010000:00010000:2.0:1789762627.275258:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000198d2894/0x416193e012b01b8a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762627.275291:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000198d2894/0x416193e012b01b8a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762627.275298:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000198d2894/0x416193e012b01b8a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762627.275304:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc2e0600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762627.275314:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000198d2894/0x416193e012b01b8a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762627.275321:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000198d2894/0x416193e012b01b8a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762627.275335:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000198d2894/0x416193e012b01b8a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762627.275341:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000198d2894/0x416193e012b01b8a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762627.275390:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014800256:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3545us (198682us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762627.275398:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 205 00000100:00100000:2.0:1789762627.275403:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 150575591378 time 150575731901 state 0x2 ref 0 00000100:00100000:2.0:1789762627.277546:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762627.481208:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762627.481336:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014800512 00000100:00100000:2.0:1789762627.481351:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014800512 00000100:00100000:2.0:1789762627.481415:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 150775731901 time 150775824868 state 0x0 ref 1 00000100:00100000:2.0:1789762627.481437:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 150975824868 time 150775824868 state 0x1 ref 1 00010000:00010000:0.0:1789762627.511277:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762627.672043:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 150975824868 time 150975946203 state 0x0 ref 1 00000100:00100000:0.0:1789762627.672072:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014800512:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762627.672133:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b01b98 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762627.672147:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762627.672150:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762627.672155:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b01b98 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762627.672166:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b01b98 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762627.675793:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b01b98 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762627.675805:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b01b98 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762627.675812:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b01b98 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762627.675817:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762627.675821:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b01b98 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762627.675828:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b01b98 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762627.675841:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b01b98 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762627.675846:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b01b98 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762627.675895:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014800512:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3836us (194693us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762627.675903:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 207 00000100:00100000:0.0:1789762627.675914:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 150975824868 time 150975946203 state 0x2 ref 0 00000100:00100000:0.0:1789762627.678232:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762627.879019:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762627.879066:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014800768 00000100:00100000:0.0:1789762627.879075:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014800768 00000100:00100000:0.0:1789762627.879130:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 151175946203 time 151176055537 state 0x0 ref 1 00000100:00100000:0.0:1789762627.879141:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 151376055537 time 151176055537 state 0x1 ref 1 00000100:00100000:2.0:1789762628.072243:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 151376055537 time 151376145365 state 0x0 ref 1 00000100:00100000:2.0:1789762628.072284:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014800768:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:0.0:1789762628.072343:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789762628.072354:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b01ba6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762628.072396:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762628.072402:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762628.072410:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b01ba6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762628.072427:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b01ba6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762628.281072:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762628.472466:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 151776237760 time 151776367093 state 0x0 ref 1 00000100:00100000:0.0:1789762628.472499:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014801024:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762628.472541:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000010fad67e/0x416193e012b01bb4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762628.472557:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762628.472561:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762628.472567:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010fad67e/0x416193e012b01bb4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762628.472581:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010fad67e/0x416193e012b01bb4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762628.476792:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000010fad67e/0x416193e012b01bb4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762628.476806:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010fad67e/0x416193e012b01bb4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762628.476829:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010fad67e/0x416193e012b01bb4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762628.476837:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762628.476841:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010fad67e/0x416193e012b01bb4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762628.476850:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010fad67e/0x416193e012b01bb4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762628.476869:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010fad67e/0x416193e012b01bb4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762628.476876:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010fad67e/0x416193e012b01bb4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762628.476942:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014801024:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4457us (195871us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762628.476950:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 211 00000100:00100000:0.0:1789762628.476955:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 151776237760 time 151776367093 state 0x2 ref 0 00000100:00100000:0.0:1789762628.479878:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762628.535217:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762628.692174:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762628.692224:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014801280 00000100:00100000:0.0:1789762628.692234:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014801280 00000100:00100000:0.0:1789762628.692284:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 151976367093 time 151976459369 state 0x0 ref 1 00000100:00100000:0.0:1789762628.692307:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 152176459369 time 151976459369 state 0x1 ref 1 00000100:00100000:0.0:1789762628.706196:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762628.706240:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762628.706259:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014801536 00000100:00100000:0.0:1789762628.706272:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762628.706275:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014801536 00000100:00080000:0.0:1789762628.706326:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762628.706348:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 69 00000100:00100000:2.0:1789762628.706361:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2366:x1876702014801536:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:0.0:1789762628.706379:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762628.706432:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902df480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2366:x1876702014801536:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 72us (240us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762628.706468:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 69 00000100:00100000:2.0:1789762628.727257:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762628.727276:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d84543b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022122624:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762628.727327:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789762628.727415:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762628.879957:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762629.078630:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762629.239263:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead6e4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022122880:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762629.239322:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb14000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022123008:0@lo:400:kworker.0 00000100:00100000:0.0:1789762629.239324:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762629.239392:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb17480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022123392:0@lo:400:kworker.0 00000100:00100000:0.0:1789762629.239446:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762629.239496:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022122880 00000100:00100000:0.0:1789762629.239501:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022122880 00000100:00100000:0.0:1789762629.239547:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 34 00000100:00100000:0.0:1789762629.239559:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022122880:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789762629.239590:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762629.239670:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022122880:12345-0@lo:13:osp-pre-1-0.0 Request processed in 114us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762629.239682:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 34 00000100:00100000:0.0:1789762629.239715:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022123392 00000100:00100000:0.0:1789762629.239720:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022123392 00000100:00100000:2.0:1789762629.239750:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762629.239759:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 40 00000100:00100000:0.0:1789762629.239769:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022123392:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762629.239826:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022123392:12345-0@lo:400:kworker.0 Request processed in 58us (377us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762629.239840:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 40 00000100:00100000:0.0:1789762629.239905:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb14000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022123008:0@lo:400:kworker.0 00000100:00100000:0.0:1789762629.272814:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 152576628600 time 152576717996 state 0x0 ref 1 00000100:00100000:0.0:1789762629.272842:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014801920:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762629.272880:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b01bd0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762629.272909:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762629.272912:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762629.272917:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b01bd0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762629.272928:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b01bd0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762629.277035:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b01bd0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762629.277049:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b01bd0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762629.277057:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b01bd0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762629.277062:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aa400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762629.277078:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b01bd0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762629.277108:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b01bd0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762629.277124:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b01bd0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762629.277132:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b01bd0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762629.277219:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014801920:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4385us (198587us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762629.277230:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 215 00000100:00100000:0.0:1789762629.277250:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 152576628600 time 152576717996 state 0x2 ref 0 00000100:00100000:0.0:1789762629.279469:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762629.478950:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762629.479034:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014802176 00000100:00100000:0.0:1789762629.479048:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014802176 00000100:00100000:0.0:1789762629.479152:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 152776717996 time 152776795115 state 0x0 ref 1 00000100:00100000:0.0:1789762629.479171:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 152976795115 time 152776795115 state 0x1 ref 1 00010000:00010000:2.0:1789762629.559347:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762629.678510:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762629.878668:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762630.073148:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 153376954765 time 153377050616 state 0x0 ref 1 00000100:00100000:0.0:1789762630.073183:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014802432:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762630.073238:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e3b1266f/0x416193e012b01bec lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762630.073251:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762630.073255:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762630.073260:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e3b1266f/0x416193e012b01bec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762630.073270:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e3b1266f/0x416193e012b01bec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762630.278974:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014802688 00000100:00100000:0.0:1789762630.278982:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014802688 00000100:00100000:0.0:1789762630.279012:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 153577050616 time 153577119185 state 0x0 ref 1 00000100:00100000:0.0:1789762630.279021:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 153777119185 time 153577119185 state 0x1 ref 1 00010000:00010000:0.0:1789762630.583179:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762630.679024:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762630.873483:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 154177270165 time 154177385348 state 0x0 ref 1 00000100:00100000:0.0:1789762630.873513:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014802944:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762630.873550:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b01c08 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762630.873565:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762630.873569:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762630.873573:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b01c08 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762630.873605:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b01c08 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762630.878460:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762631.079070:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762631.279741:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762631.473806:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 154777610715 time 154777704663 state 0x0 ref 1 00000100:00100000:2.0:1789762631.473854:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbe1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014803328:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762631.473904:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b01c1d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762631.473922:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762631.473926:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762631.473932:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b01c1d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762631.473945:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b01c1d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762631.479821:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762631.607195:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762631.679358:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762631.879770:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:0.0:1789762632.074338:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762632.081492:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762632.274283:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 155578077570 time 155578179376 state 0x0 ref 1 00000100:00100000:2.0:1789762632.274328:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014803840:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762632.274381:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002ab85522/0x416193e012b01c39 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762632.274399:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762632.274402:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762632.274408:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002ab85522/0x416193e012b01c39 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762632.274422:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002ab85522/0x416193e012b01c39 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762632.278003:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002ab85522/0x416193e012b01c39 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762632.278013:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002ab85522/0x416193e012b01c39 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762632.278020:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002ab85522/0x416193e012b01c39 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762632.278026:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e88523200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762632.278042:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002ab85522/0x416193e012b01c39 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762632.278050:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002ab85522/0x416193e012b01c39 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762632.278065:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002ab85522/0x416193e012b01c39 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762632.278070:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002ab85522/0x416193e012b01c39 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762632.278158:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014803840:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3852us (196685us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762632.278165:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 230 00000100:00100000:2.0:1789762632.278170:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 155578077570 time 155578179376 state 0x2 ref 0 00000100:00100000:2.0:1789762632.280157:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762632.474391:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 155778179376 time 155778289925 state 0x0 ref 1 00000100:00100000:0.0:1789762632.474433:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014803968:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762632.474500:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b01c40 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762632.474521:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762632.474553:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762632.474561:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b01c40 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762632.474580:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b01c40 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762632.478412:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b01c40 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762632.478425:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b01c40 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762632.478450:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b01c40 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762632.478457:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812ca00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762632.478461:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b01c40 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762632.478469:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b01c40 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762632.478499:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b01c40 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762632.478505:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b01c40 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762632.478566:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014803968:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4154us (198409us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762632.478573:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 231 00000100:00100000:0.0:1789762632.478580:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 155778179376 time 155778289925 state 0x2 ref 0 00000100:00100000:2.0:1789762632.481164:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762632.631215:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762632.679834:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762632.679888:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014804224 00000100:00100000:0.0:1789762632.679899:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014804224 00000100:00100000:0.0:1789762632.679942:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 155978289925 time 155978357753 state 0x0 ref 1 00000100:00100000:0.0:1789762632.679953:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 156178357753 time 155978357753 state 0x1 ref 1 00000100:00100000:0.0:1789762632.892505:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762633.079199:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01c55 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762633.079216:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01c55 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762633.079224:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01c55 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762633.079246:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa64e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762633.079250:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01c55 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762633.079262:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01c55 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762633.079286:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01c55 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762633.079296:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096d771ca/0x416193e012b01c55 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x5:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762633.079389:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014804352:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4675us (186882us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762633.079399:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 234 00000100:00100000:2.0:1789762633.079406:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf520600 key uid={500} rule@0xffff981e97832900 gen 0 rate 5 depth 3 token 0 deadline 156378492768 time 156378597180 state 0x2 ref 0 00000100:00100000:0.0:1789762633.080974:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:0.0:1789762633.081187:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762633.128764:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762633.141989:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762633.146517:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762633.148668:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762633.153409:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762633.154721:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762633.157599:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762633.159400:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762633.159414:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b01ca2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762633.159430:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfcc00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762633.159437:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b01ca2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762633.159469:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b01ca2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762633.159498:0:5793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff981ec2179700) 00010000:00010000:2.0:1789762633.159504:0:5793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789762633.159511:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b01ca2 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762633.159527:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b01ca2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xb:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762633.159567:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762633.159571:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762633.159574:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762633.160782:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762633.161971:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762633.655193:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762633.826265:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762633.826304:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762633.826339:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762633.826344:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014807040 00000100:00100000:0.0:1789762633.826351:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014807040 00000100:00100000:2.0:1789762633.826377:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762633.826401:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:2.0:1789762633.826408:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014806912 00000100:00100000:0.0:1789762633.826414:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014807040:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762633.826416:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014806912 00000100:00100000:2.0:1789762633.826445:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014807168 00000100:00100000:2.0:1789762633.826449:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014807168 00000100:00100000:2.0:1789762633.826476:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 74 00000100:00100000:0.0:1789762633.826490:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014807040:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 76us (179us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762633.826492:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702014806912:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762633.826504:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:2.0:1789762633.826542:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702014806912:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 58us (204us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762633.826550:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 74 00000100:00080000:2.0:1789762633.826585:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762633.847388:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022123520 00000100:00100000:0.0:1789762633.847395:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022123520 00000100:00100000:0.0:1789762633.847447:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 35 00000100:00100000:0.0:1789762633.847456:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022123520:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762633.847484:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762633.847537:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022123520:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (249us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762633.847544:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 35 00000100:00100000:2.0:1789762633.847572:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc2d1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022123520:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762634.047482:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022123648 00000100:00100000:0.0:1789762634.047492:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022123648 00000100:00100000:0.0:1789762634.047540:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 56 00000100:00100000:0.0:1789762634.047548:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022123648:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1789762634.047573:0:6649:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:5 count 1 00002000:00010000:0.0:1789762634.047598:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x5:0x0] initial LVB size: 104857600, mtime: 0x6aad9c35, atime: 0x6aad9c21, ctime: 0x6aad9c35, blocks: 0x3200b 00010000:00010000:0.0:1789762634.047621:0:6649:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01cbe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762634.047635:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=157s v=(0 0) 00010000:00010000:0.0:1789762634.047639:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762634.047644:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01cbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762634.047656:0:6649:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01cbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762634.047663:0:6649:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01cbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762634.047669:0:6649:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042345ac7/0x416193e012b01cbe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762634.047685:0:6649:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1789762634.048080:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022123648:12345-0@lo:6:osp-syn-0-0.0 Request processed in 533us (629us total) trans 4294967427 rc 0/0 00000100:00100000:0.0:1789762634.048112:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 56 00000004:00080000:0.0:1789762634.048247:0:11902:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 157352159167ns, next at 127943141165ns, rc = 0 00000100:00100000:0.0:1789762634.360204:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789762634.360221:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d831a9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022123776:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762634.360279:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762634.360339:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac59c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022124416:0@lo:400:kworker.0 00000100:00100000:0.0:1789762634.360372:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762634.360426:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022123776 00000100:00100000:0.0:1789762634.360435:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022123776 00000100:00100000:0.0:1789762634.360521:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 36 00000100:00100000:0.0:1789762634.360555:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022123776:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789762634.360586:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762634.360679:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022123776:12345-0@lo:13:osp-pre-1-0.0 Request processed in 125us (396us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762634.360694:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 36 00000100:00100000:0.0:1789762634.360722:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022124416 00000100:00100000:0.0:1789762634.360728:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022124416 00000100:00100000:0.0:1789762634.360761:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 42 00000100:00100000:0.0:1789762634.360793:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022124416:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762634.360862:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022124416:12345-0@lo:400:kworker.0 Request processed in 71us (488us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762634.360871:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 42 00010000:00010000:2.0:1789762634.679270:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762636.727262:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762638.775259:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762638.946710:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762638.946775:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762638.946811:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762638.946875:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762638.946891:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014807552 00000100:00100000:2.0:1789762638.946900:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014807552 00000100:00100000:0.0:1789762638.946962:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014807680 00000100:00100000:0.0:1789762638.947011:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014807680 00000100:00100000:2.0:1789762638.947029:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 43 00000100:00100000:2.0:1789762638.947049:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014807552:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762638.947182:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 80 00000100:00100000:0.0:1789762638.947210:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014807680:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762638.947225:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014807552:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 178us (406us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762638.947239:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 43 00000100:00100000:0.0:1789762638.947307:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831ab800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014807680:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 99us (430us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762638.947332:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 80 00000100:00100000:2.0:1789762638.967474:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022124544 00000100:00100000:2.0:1789762638.967481:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022124544 00000100:00080000:0.0:1789762638.967543:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762638.967560:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 37 00000100:00100000:2.0:1789762638.967586:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84543800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022124544:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762638.967625:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762638.967688:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022124544:12345-0@lo:13:osp-pre-0-0.0 Request processed in 105us (286us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762638.967698:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 37 00000001:00080000:2.0:1789762638.994464:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967427 is committed 00000100:00100000:0.0:1789762639.479503:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d831a9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022125056:0@lo:400:kworker.0 00000100:00100000:0.0:1789762639.479644:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762639.479817:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d831aa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022124928:0@lo:400:kworker.0 00000100:00100000:0.0:1789762639.479949:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022124800 00000100:00100000:0.0:1789762639.479967:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022124800 00000100:00100000:0.0:1789762639.480195:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 38 00000100:00100000:0.0:1789762639.480229:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022124800:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789762639.480300:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762639.480415:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022124800:12345-0@lo:13:osp-pre-1-0.0 Request processed in 197us (819us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762639.480431:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 38 00000100:00080000:0.0:1789762639.480491:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762639.480504:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762639.480552:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762639.480691:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022124928 00000100:00100000:0.0:1789762639.480702:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022124928 00000100:00100000:0.0:1789762639.480788:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 83 00000100:00100000:0.0:1789762639.480826:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022124928:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762639.480989:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022124928:12345-0@lo:400:kworker.0 Request processed in 175us (435us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762639.481043:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 83 00010000:00010000:0.0:1789762640.823187:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762642.871262:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762644.066428:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762644.066471:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762644.066503:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762644.066546:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762644.066586:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014808064 00000100:00100000:2.0:1789762644.066592:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014808064 00000100:00100000:2.0:1789762644.066649:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 45 00000100:00100000:2.0:1789762644.066664:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014808064:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762644.066729:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014808064:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 66us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762644.066752:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 45 00000100:00100000:2.0:1789762644.087272:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc2d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022125440:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762644.087335:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1789762644.087416:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762644.087617:0:3274:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967427 gen 2 00000004:00080000:2.0:1789762644.087624:0:3274:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebc2d1880, transno 4294967427 00000100:00100000:2.0:1789762644.087652:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc2d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022125440:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762644.599312:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc2d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022126080:0@lo:400:kworker.0 00000100:00100000:2.0:1789762644.599394:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1789762644.599493:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762644.599508:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762644.599614:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 87 00000100:00100000:2.0:1789762644.599632:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022125952:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762644.599635:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022125568 00000100:00100000:0.0:1789762644.599643:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022125568 00000100:00100000:0.0:1789762644.599692:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 40 00000100:00100000:2.0:1789762644.599700:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022125952:12345-0@lo:400:kworker.0 Request processed in 75us (191us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762644.599702:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022125568:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762644.599713:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 87 00002000:00000024:0.0:1789762644.599740:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762644.599746:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc2d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022125824:0@lo:400:kworker.0 00000100:00100000:2.0:1789762644.599790:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762644.599797:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022125568:12345-0@lo:13:osp-pre-1-0.0 Request processed in 95us (393us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762644.599807:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 40 00000100:00100000:2.0:1789762644.599822:0:3272:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789762644.599833:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc2d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022126208:0@lo:400:kworker.0 00000100:00100000:2.0:1789762644.599873:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762644.599928:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022126208 00000100:00100000:2.0:1789762644.599935:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022126208 00000100:00100000:2.0:1789762644.599983:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 46 00000100:00100000:2.0:1789762644.599997:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022126208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762644.600040:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022126208:12345-0@lo:400:kworker.0 Request processed in 48us (169us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762644.600048:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 46 00000100:00100000:2.0:1789762644.600111:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc2d0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022126208:0@lo:400:kworker.0 00000100:00100000:2.0:1789762644.600124:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc2d1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022125824:0@lo:400:kworker.0 00000100:00100000:2.0:1789762644.600172:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc2d0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022126080:0@lo:400:kworker.0 00000100:00100000:2.0:1789762644.600203:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc2d3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022125952:0@lo:400:kworker.0 00010000:00010000:0.0:1789762644.919276:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762646.967287:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762649.015245:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762649.186684:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762649.186737:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762649.186763:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762649.186810:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762649.186857:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014808704 00000100:00100000:0.0:1789762649.186862:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014808576 00000100:00100000:2.0:1789762649.186864:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014808704 00000100:00100000:0.0:1789762649.186871:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014808576 00000100:00100000:2.0:1789762649.186937:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 90 00000100:00100000:2.0:1789762649.186953:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014808704:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762649.186971:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 47 00000100:00100000:0.0:1789762649.186990:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014808576:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762649.187025:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014808704:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 76us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762649.187038:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014808576:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 53us (269us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762649.187050:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 47 00000100:00100000:2.0:1789762649.187051:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 90 00000100:00100000:2.0:1789762649.207340:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d84542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022126336:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762649.207420:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789762649.207655:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762649.207923:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d84542a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022126336:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762649.719363:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc2d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022126720:0@lo:400:kworker.0 00000100:00100000:2.0:1789762649.719379:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc2d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022126464:0@lo:400:kworker.0 00000100:00100000:0.0:1789762649.719418:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762649.719471:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762649.719505:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc2d0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022127232:0@lo:400:kworker.0 00000100:00100000:2.0:1789762649.719549:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc2d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022126976:0@lo:400:kworker.0 00000100:00100000:0.0:1789762649.719559:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762649.719618:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762649.719625:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762649.719635:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc2d1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022126848:0@lo:400:kworker.0 00000100:00100000:0.0:1789762649.719702:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762649.719709:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022126464 00000100:00100000:2.0:1789762649.719717:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022126464 00000100:00100000:0.0:1789762649.719773:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022126720 00000100:00100000:0.0:1789762649.719782:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022126720 00000100:00100000:2.0:1789762649.719848:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:0.0:1789762649.719852:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022127104 00000100:00100000:0.0:1789762649.719860:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022127104 00000100:00100000:2.0:1789762649.719868:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85086450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022126464:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762649.719922:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022126848 00000100:00100000:0.0:1789762649.719930:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022126848 00000100:00100000:2.0:1789762649.719966:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022127232 00000100:00100000:2.0:1789762649.719971:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022127232 00000100:00100000:0.0:1789762649.720015:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 92 00000100:00100000:2.0:1789762649.720025:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022126976 00000100:00100000:2.0:1789762649.720029:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022126976 00000100:00100000:0.0:1789762649.720037:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022127104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762649.720076:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 48 00000100:00100000:2.0:1789762649.720134:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022127232:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762649.720189:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022127104:12345-0@lo:400:kworker.0 Request processed in 161us (592us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762649.720215:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 92 00000100:00100000:2.0:1789762649.720215:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022127232:12345-0@lo:400:kworker.0 Request processed in 126us (655us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762649.720257:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 48 00000100:00100000:0.0:1789762649.720287:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 49 00000100:00100000:0.0:1789762649.720315:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022126976:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762649.720371:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85086450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022126464:12345-0@lo:400:kworker.0 Request processed in 511us (902us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762649.720383:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00100000:0.0:1789762649.720399:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022126976:12345-0@lo:400:kworker.0 Request processed in 88us (777us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762649.720411:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 49 00000100:00100000:2.0:1789762649.720480:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc2d0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022126464:0@lo:400:kworker.0 00000100:00100000:2.0:1789762649.720521:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc2d2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022126976:0@lo:400:kworker.0 00010000:00010000:2.0:1789762651.063188:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762653.111231:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:0.0:1789762654.342969:0:6674:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:0.0:1789762654.342977:0:6674:0:(genops.c:885:class_export_put()) final put ffff981ebc295800/f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:0.0:1789762654.343032:0:37:0:(genops.c:844:class_export_destroy()) destroying export ffff981ebc295800/f1bd28de-ee78-4d18-bad3-fa7cec968893 for lustre-OST0000 00000020:00000080:0.0:1789762654.343125:0:37:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff981ebc290000 for lustre-OST0000 00000100:00100000:2.0:1789762654.839306:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfed2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022127488:0@lo:400:kworker.0 00000100:00100000:2.0:1789762654.839409:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762654.839506:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc426a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022127872:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762654.839557:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762654.839625:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022127488 00000100:00100000:2.0:1789762654.839635:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022127488 00000100:00100000:2.0:1789762654.839688:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 243 00000100:00100000:2.0:1789762654.839706:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85087c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022127488:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762654.839782:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85087c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022127488:12345-0@lo:400:kworker.0 Request processed in 84us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762654.839798:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 243 00000100:00100000:2.0:1789762654.839858:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022127872 00000100:00100000:2.0:1789762654.839866:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022127872 00000100:00100000:2.0:1789762654.839936:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 44 00000100:00100000:2.0:1789762654.839966:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022127872:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762654.839999:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762654.840069:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022127872:12345-0@lo:13:osp-pre-1-0.0 Request processed in 117us (511us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762654.840142:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 44 00000100:00100000:2.0:1789762654.840218:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfed2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022127488:0@lo:400:kworker.0 00000100:00100000:2.0:1789762654.840271:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc426a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022127872:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789762655.160201:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762656.183202:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762659.105230:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014809472 00000100:00100000:0.0:1789762659.105238:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014809472 00000100:00100000:0.0:1789762659.105285:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014809728 00000100:00100000:0.0:1789762659.105290:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014809728 00000100:00100000:2.0:1789762659.105304:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 90 00000100:00100000:2.0:1789762659.105320:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2364:x1876702014809472:12345-192.168.203.48@tcp:41:mkdir.0 00000004:00000024:2.0:1789762659.105354:0:6611:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:2.0:1789762659.105410:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2364:x1876702014809472:12345-192.168.203.48@tcp:41:mkdir.0 Request processed in 92us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762659.105440:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 90 00010000:00010000:0.0:1789762659.107867:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64a00/0x416193e012b01cda lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.107878:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64a00/0x416193e012b01cda lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.107883:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64a00/0x416193e012b01cda lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.107888:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa64a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762659.107892:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64a00/0x416193e012b01cda lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.107897:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64a00/0x416193e012b01cda lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.107910:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981e83a46000) 00010000:00010000:0.0:1789762659.107912:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762659.107916:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64a00/0x416193e012b01cda lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.107921:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64a00/0x416193e012b01cda lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762659.113460:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.114998:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014810112 00000100:00100000:0.0:1789762659.115012:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014810112 00000100:00100000:0.0:1789762659.115052:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 93 00000100:00100000:0.0:1789762659.115062:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:11164:x1876702014810112:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762659.115079:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762659.115082:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1789762659.115153:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65c00/0x416193e012b01ce8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0d6 expref: 7 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115180:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46600) 00010000:00010000:0.0:1789762659.115196:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762659.115210:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812cc00/0x416193e012b01cef lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd0dd expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115339:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01cf6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115361:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01cf6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115370:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01cf6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115385:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01cfd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115390:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01cfd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115396:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01cfd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115624:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01d04 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115637:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01d04 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115645:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01d04 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762659.115665:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd:0x0]: returning size 16384 00010000:00010000:0.0:1789762659.115681:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01d04 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115687:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01d04 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115693:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01d04 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115697:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812de00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762659.115700:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01d04 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115705:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01d04 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115714:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762659.115716:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762659.115718:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01d04 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115739:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01d04 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115756:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01cf6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115760:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01cf6 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115766:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01cf6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115779:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762659.115783:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01cf6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115791:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01cf6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115802:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762659.115805:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762659.115809:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01cf6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115817:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01cf6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115832:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01cfd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115836:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01cfd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115841:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01cfd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115853:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812ce00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762659.115856:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01cfd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115861:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01cfd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115878:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762659.115879:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762659.115882:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01cfd lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115886:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01cfd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115916:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812cc00/0x416193e012b01cef lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0dd expref: 7 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115925:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762659.115927:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762659.115932:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812cc00/0x416193e012b01cef lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0dd expref: 7 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.115954:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8812cc00, rc 0) 00000100:00100000:0.0:1789762659.115995:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:11164:x1876702014810112:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 935us (1047us total) trans 0 rc 301/301 00000100:00100000:0.0:1789762659.116001:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 93 00000100:00100000:2.0:1789762659.116939:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.118252:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014810496 00000100:00100000:0.0:1789762659.118259:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014810496 00000100:00100000:0.0:1789762659.118295:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 95 00000100:00100000:0.0:1789762659.118307:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:11164:x1876702014810496:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762659.118316:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762659.118326:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812da00/0x416193e012b01d19 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd0eb expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118388:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01d20 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118401:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01d20 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118408:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01d20 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118432:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01d27 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118438:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01d27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118445:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01d27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762659.118524:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd:0x0]: returning size 16384 00010000:00010000:0.0:1789762659.118574:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01d20 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118580:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01d20 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118584:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01d20 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118605:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762659.118608:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01d20 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118627:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01d20 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118637:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:0.0:1789762659.118639:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762659.118642:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01d20 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118647:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01d20 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118655:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01d27 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118660:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01d27 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118664:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01d27 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118668:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762659.118670:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01d27 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118674:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01d27 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118679:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981ea0aa3800) 00010000:00010000:0.0:1789762659.118681:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762659.118683:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01d27 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118696:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01d27 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118723:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812da00/0x416193e012b01d19 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0eb expref: 9 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118748:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:0.0:1789762659.118749:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762659.118752:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812da00/0x416193e012b01d19 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0eb expref: 9 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.118761:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8812da00, rc 0) 00000100:00100000:0.0:1789762659.118802:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:11164:x1876702014810496:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 498us (599us total) trans 4294967360 rc 301/301 00000100:00100000:0.0:1789762659.118812:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 95 00000100:00100000:0.0:1789762659.119830:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.119873:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014810752 00000100:00100000:2.0:1789762659.119880:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014810752 00000100:00100000:2.0:1789762659.119913:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 96 00000100:00100000:2.0:1789762659.119921:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:11164:x1876702014810752:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:2.0:1789762659.119952:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d35 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.119960:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d35 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.119966:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d35 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762659.120058:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000091e08523 reply state 00000000be326e30 transno 4294967361 ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d35 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762659.120064:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762659.120068:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd:0x0]: returning size 16384 00000100:00100000:2.0:1789762659.120172:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:11164:x1876702014810752:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 250us (340us total) trans 4294967361 rc 0/0 00000100:00100000:2.0:1789762659.120182:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 96 00000100:00100000:0.0:1789762659.121036:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.125657:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.127309:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762659.128285:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64200/0x416193e012b01d58 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.128294:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64200/0x416193e012b01d58 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.128310:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64200/0x416193e012b01d58 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.128314:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa64200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762659.128317:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64200/0x416193e012b01d58 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.128322:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64200/0x416193e012b01d58 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.128330:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:2.0:1789762659.128333:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762659.128336:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64200/0x416193e012b01d58 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.128342:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64200/0x416193e012b01d58 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762659.144725:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014811520 00000100:00100000:2.0:1789762659.144734:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014811520 00000100:00100000:2.0:1789762659.144796:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 99 00000100:00100000:2.0:1789762659.144810:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:11169:x1876702014811520:12345-192.168.203.48@tcp:101:touch.500 00010000:00010000:2.0:1789762659.144828:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762659.144842:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b01d5f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd0f9 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.144885:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d66 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.144914:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d66 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.144933:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d66 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.144946:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01d6d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.144953:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01d6d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.144960:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01d6d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145132:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b01d74 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145141:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b01d74 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145177:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b01d74 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762659.145196:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd:0x0]: returning size 16384 00000004:00200000:2.0:1789762659.145200:0:5805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762659.145236:0:5805:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b01d74 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145248:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d66 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145254:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d66 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145273:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d66 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145279:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762659.145283:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d66 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145291:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d66 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145302:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:2.0:1789762659.145306:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762659.145309:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d66 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145316:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b01d66 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145327:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01d6d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145333:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01d6d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145339:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01d6d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145345:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762659.145348:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01d6d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145380:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01d6d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145387:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981e82a26000) 00010000:00010000:2.0:1789762659.145390:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762659.145392:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01d6d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145397:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b01d6d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145439:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b01d5f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0f9 expref: 13 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145451:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b01d74 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0f9 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.145458:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff981e82a26a00) 00010000:00010000:2.0:1789762659.145477:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789762659.145480:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81483c00, rc 0) 00000100:00100000:2.0:1789762659.145533:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:11169:x1876702014811520:12345-192.168.203.48@tcp:101:touch.500 Request processed in 726us (853us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762659.145544:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 99 00000100:00100000:2.0:1789762659.146964:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014811776 00000100:00100000:2.0:1789762659.146971:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014811776 00000100:00100000:2.0:1789762659.147032:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 100 00000100:00100000:2.0:1789762659.147050:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:11169:x1876702014811776:12345-192.168.203.48@tcp:101:touch.500 00010000:00010000:2.0:1789762659.147073:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762659.147083:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b01d7b lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd100 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.147177:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01d82 lrc: 3/0,1 mode: --/CW res: [0x200000401:0xd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.147186:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01d82 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.147218:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01d82 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.147292:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b01d89 lrc: 3/0,1 mode: --/PW res: [0x200000401:0xd:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.147299:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b01d89 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.147305:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b01d89 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762659.147932:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x6:0x0], index 0: 0 00000004:00080000:2.0:1789762659.147941:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x4:0x0], index 1: 0 00010000:00010000:2.0:1789762659.148148:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b01d90 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.148156:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b01d90 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.148161:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b01d90 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.148236:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01d82 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.148244:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01d82 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.148251:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01d82 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.148258:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762659.148261:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01d82 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.148267:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01d82 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.148277:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff981e82a26a00) 00010000:00010000:2.0:1789762659.148281:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762659.148284:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01d82 lrc: 2/0,0 mode: --/CW res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.148291:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b01d82 lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762659.148304:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b378392e reply state 00000000147b6086 transno 4294967364 ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b01d89 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762659.148311:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762659.148335:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b01d7b lrc: 0/0,0 mode: --/CW res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd100 expref: 15 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.148380:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b01d90 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd100 expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.148387:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xf:0x0].0x0 (ffff981e82a26500) 00010000:00010000:2.0:1789762659.148390:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762659.148392:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81482000, rc 0) 00000100:00100000:2.0:1789762659.148466:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:11169:x1876702014811776:12345-192.168.203.48@tcp:101:touch.500 Request processed in 1409us (1571us total) trans 4294967364 rc 0/0 00000100:00100000:2.0:1789762659.148472:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 100 00010000:00010000:2.0:1789762659.150908:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64000/0x416193e012b01d9e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.150916:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64000/0x416193e012b01d9e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.150921:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64000/0x416193e012b01d9e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.150927:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa64000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762659.150931:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64000/0x416193e012b01d9e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.150936:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64000/0x416193e012b01d9e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.150948:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xf:0x0].0x0 (ffff981e83a46100) 00010000:00010000:2.0:1789762659.150952:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762659.150957:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64000/0x416193e012b01d9e lrc: 2/0,0 mode: --/PW res: [0x200000401:0xf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.150970:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64000/0x416193e012b01d9e lrc: 0/0,0 mode: --/PW res: [0x200000401:0xf:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762659.151209:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014812160 00000100:00100000:2.0:1789762659.151216:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014812160 00000100:00100000:2.0:1789762659.151266:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 102 00000100:00100000:2.0:1789762659.151277:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:11169:x1876702014812160:12345-192.168.203.48@tcp:101:touch.500 00010000:00010000:2.0:1789762659.151289:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762659.151301:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01da5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd107 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.151327:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01da5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xf:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd107 expref: 15 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.151338:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01da5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xf:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd107 expref: 15 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.151352:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xf:0x0].0x0 (ffff981e82a26900) 00010000:00010000:2.0:1789762659.151357:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789762659.151361:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac31400, rc 0) 00000100:00100000:2.0:1789762659.151406:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:11169:x1876702014812160:12345-192.168.203.48@tcp:101:touch.500 Request processed in 130us (234us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762659.151426:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 102 00000100:00100000:2.0:1789762659.152455:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.152526:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.152586:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014812288 00000100:00100000:0.0:1789762659.152587:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014812544 00000100:00100000:2.0:1789762659.152590:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014812288 00000100:00100000:0.0:1789762659.152594:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014812544 00000100:00100000:0.0:1789762659.152646:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 58 00000100:00100000:0.0:1789762659.152660:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:0+-99:2362:x1876702014812544:12345-192.168.203.48@tcp:8:touch.500 00010000:00080000:0.0:1789762659.152689:0:10916:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp t0 exp 0000000000000000 cur 182 last 0 00000020:00000080:0.0:1789762659.152732:0:10916:0:(genops.c:1385:class_connect()) connect: client f1bd28de-ee78-4d18-bad3-fa7cec968893, cookie 0x416193e012b01dac 00002000:00100000:0.0:1789762659.152747:0:10916:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli f1bd28de-ee78-4d18-bad3-fa7cec968893/ffff981ebc4c8000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:0.0:1789762659.152778:0:10916:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:0.0:1789762659.153215:0:10916:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:0.0:1789762659.153240:0:10916:0:(import.c:65:import_set_state_nolock()) ffff981ebc4c9800 : changing import state from RECOVER to FULL 00000100:00100000:0.0:1789762659.153289:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2362:x1876702014812544:12345-192.168.203.48@tcp:8:touch.500 Request processed in 628us (762us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762659.153315:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 58 00000100:00100000:2.0:1789762659.154215:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.154254:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:0.0:1789762659.154320:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762659.154329:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:00100000:2.0:1789762659.172635:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000001:00080000:2.0:1789762659.172662:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967322 is committed 00000100:00100000:0.0:1789762659.172748:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:0:x1876702014812672:12345-192.168.203.48@tcp:2:touch.500 Request processed in 18374us (18523us total) trans 4294967322 rc 0/0 00000100:00100000:0.0:1789762659.172761:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 60 00000100:00100000:0.0:1789762659.174880:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:0:x1876702014812416:12345-192.168.203.48@tcp:2:touch.500 Request processed in 20491us (20620us total) trans 4294967428 rc 0/0 00000100:00100000:0.0:1789762659.174893:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 61 00000100:00100000:2.0:1789762659.175940:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.198263:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.201161:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762659.202116:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65800/0x416193e012b01ddd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.202144:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65800/0x416193e012b01ddd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.202151:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65800/0x416193e012b01ddd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.202157:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa65800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762659.202161:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65800/0x416193e012b01ddd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.202167:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65800/0x416193e012b01ddd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.202192:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x10:0x0].0x0 (ffff981e83a46b00) 00010000:00010000:0.0:1789762659.202195:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762659.202200:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65800/0x416193e012b01ddd lrc: 2/0,0 mode: --/PW res: [0x200000401:0x10:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.202206:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65800/0x416193e012b01ddd lrc: 0/0,0 mode: --/PW res: [0x200000401:0x10:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762659.202254:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.202292:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014813568 00000100:00100000:0.0:1789762659.202298:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014813568 00000100:00100000:0.0:1789762659.202336:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 105 00000100:00100000:0.0:1789762659.202346:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:11173:x1876702014813568:12345-192.168.203.48@tcp:101:touch.500 00010000:00010000:0.0:1789762659.202359:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762659.202370:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01de4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x10:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd115 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.202400:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01de4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x10:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd115 expref: 19 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.202411:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c600/0x416193e012b01de4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x10:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd115 expref: 19 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.202445:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x10:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762659.202448:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789762659.202450:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8812c600, rc 0) 00000100:00100000:0.0:1789762659.202502:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:11173:x1876702014813568:12345-192.168.203.48@tcp:101:touch.500 Request processed in 158us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762659.202519:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 105 00000100:00100000:2.0:1789762659.203302:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.203341:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014813696 00000100:00100000:0.0:1789762659.203347:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014813696 00000100:00100000:0.0:1789762659.203381:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 62 00000100:00100000:0.0:1789762659.203390:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014813696:12345-192.168.203.48@tcp:2:touch.500 00000100:00100000:2.0:1789762659.203523:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.203662:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014813696:12345-192.168.203.48@tcp:2:touch.500 Request processed in 269us (356us total) trans 4294967323 rc 0/0 00000100:00100000:0.0:1789762659.203671:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 62 00000100:00100000:2.0:1789762659.204832:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.223340:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.225067:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.227311:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.230804:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.236200:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.236248:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014814976 00000100:00100000:0.0:1789762659.236256:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014814976 00000100:00100000:0.0:1789762659.236299:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 108 00000100:00100000:0.0:1789762659.236308:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf565500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:11177:x1876702014814976:12345-192.168.203.48@tcp:101:chmod.500 00010000:00010000:0.0:1789762659.236325:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762659.236339:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01e07 lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd12a expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236385:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01e0e lrc: 3/1,0 mode: --/CR res: [0x200000401:0xd:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236396:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01e0e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236404:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01e0e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236427:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e15 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236442:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e15 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236448:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e15 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236517:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01e1c lrc: 3/1,0 mode: --/PR res: [0x200000401:0xf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236523:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01e1c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xf:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236528:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01e1c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xf:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762659.236549:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xf:0x0]: returning size 0 00000004:00200000:0.0:1789762659.236553:0:5805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762659.236581:0:5805:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01e1c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xf:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236590:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01e1c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236603:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01e0e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236611:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01e0e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236616:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01e0e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236627:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812ce00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762659.236631:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01e0e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236636:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01e0e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236646:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff981e82a26a00) 00010000:00010000:0.0:1789762659.236648:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762659.236651:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01e0e lrc: 2/0,0 mode: --/CR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236655:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b01e0e lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236664:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e15 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236668:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e15 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236672:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e15 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236675:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762659.236677:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e15 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236681:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e15 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236692:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x77289514 (ffff981ea0aa3800) 00010000:00010000:0.0:1789762659.236693:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762659.236695:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e15 lrc: 2/0,0 mode: --/PR res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236699:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e15 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236722:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d000/0x416193e012b01e07 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd12a expref: 25 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236732:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812de00/0x416193e012b01e1c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd12a expref: 24 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.236737:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xf:0x0].0x0 (ffff981e82a26900) 00010000:00010000:0.0:1789762659.236738:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762659.236740:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8812de00, rc 0) 00000100:00100000:0.0:1789762659.236780:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf565500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:11177:x1876702014814976:12345-192.168.203.48@tcp:101:chmod.500 Request processed in 474us (586us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762659.236785:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 108 00000100:00100000:2.0:1789762659.237991:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.238025:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014815104 00000100:00100000:0.0:1789762659.238031:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014815104 00000100:00100000:0.0:1789762659.238069:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 64 00000100:00100000:0.0:1789762659.238078:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf566300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014815104:12345-192.168.203.48@tcp:101:chmod.500 00010000:00010000:0.0:1789762659.238106:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1789762659.238112:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762659.238119:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b01e23 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd131 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762659.238152:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x6:0x0] initial LVB size: 0, mtime: 0x6aad9c63, atime: 0x6aad9c63, ctime: 0x6aad9c63, blocks: 0x1 00010000:00010000:0.0:1789762659.238176:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x6:0x0].0x0 since=182s v=(0 0) 00010000:00010000:0.0:1789762659.238185:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762659.238202:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b01e23 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd131 expref: 7 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.238223:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b01e23 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd131 expref: 7 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.238235:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8812d400, rc 0) 00000100:00100000:0.0:1789762659.238287:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf566300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2365:x1876702014815104:12345-192.168.203.48@tcp:101:chmod.500 Request processed in 204us (289us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762659.238310:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 64 00000100:00100000:2.0:1789762659.239488:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.240582:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.240715:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.240785:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 66 00000100:00100000:0.0:1789762659.240804:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2365:x1876702014815616:12345-192.168.203.48@tcp:2:chmod.500 00002000:00010000:0.0:1789762659.240989:0:6881:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x6:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1789762659.240995:0:6881:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x6:0x0] updating lvb mtime from disk: 1789762659 -> 1789762659 00002000:00010000:0.0:1789762659.240998:0:6881:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x6:0x0] updating lvb atime from disk: 1789762659 -> 1789762659 00002000:00010000:0.0:1789762659.241001:0:6881:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x6:0x0] updating lvb ctime from disk: 1789762659 -> 1789762659 00002000:00010000:0.0:1789762659.241005:0:6881:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x6:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:0.0:1789762659.241061:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702014815616:12345-192.168.203.48@tcp:2:chmod.500 Request processed in 262us (475us total) trans 4294967430 rc 0/0 00000100:00100000:0.0:1789762659.241132:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 66 00000100:00100000:2.0:1789762659.242006:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.243380:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.243550:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.243603:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014816128 00000100:00100000:0.0:1789762659.243612:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014816128 00000100:00100000:0.0:1789762659.243653:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 69 00000100:00100000:0.0:1789762659.243661:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf567b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014816128:12345-192.168.203.48@tcp:101:chmod.500 00010000:00010000:0.0:1789762659.243671:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762659.243689:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b01e5b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd14d expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762659.243711:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x7:0x0] initial LVB size: 0, mtime: 0x6aad9c63, atime: 0x6aad9c63, ctime: 0x6aad9c63, blocks: 0x1 00010000:00010000:0.0:1789762659.243726:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x7:0x0].0x0 since=182s v=(0 0) 00010000:00010000:0.0:1789762659.243731:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762659.243741:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b01e5b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd14d expref: 9 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.243754:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b01e5b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd14d expref: 9 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.243762:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8812d000, rc 0) 00000100:00100000:0.0:1789762659.243793:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf567b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702014816128:12345-192.168.203.48@tcp:101:chmod.500 Request processed in 134us (257us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762659.243800:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 69 00000100:00100000:2.0:1789762659.244709:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.244750:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014816384 00000100:00100000:0.0:1789762659.244756:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014816384 00000100:00100000:0.0:1789762659.244781:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 111 00000100:00100000:0.0:1789762659.244786:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf566d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:11177:x1876702014816384:12345-192.168.203.48@tcp:36:chmod.500 00010000:00010000:0.0:1789762659.244808:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762659.244826:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa64400/0x416193e012b01e4d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd13f expref: 25 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.244837:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x10:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762659.244840:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762659.244852:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e62 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x10:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.244858:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e62 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.244864:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e62 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762659.244978:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f2f0085b reply state 00000000f3b06733 transno 4294967373 ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e62 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762659.244984:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762659.245027:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf566d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:11177:x1876702014816384:12345-192.168.203.48@tcp:36:chmod.500 Request processed in 240us (318us total) trans 4294967373 rc 0/0 00000100:00100000:0.0:1789762659.245032:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 111 00010000:00010000:0.0:1789762659.245530:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e62 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.245555:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e62 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.245559:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e62 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.245562:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762659.245571:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e62 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.245575:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e62 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.245582:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x10:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762659.245583:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762659.245586:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e62 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x10:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.245590:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c400/0x416193e012b01e62 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x10:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762659.245789:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.245813:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.245826:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014816512 00000100:00100000:0.0:1789762659.245831:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014816512 00000100:00100000:0.0:1789762659.245854:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014816640 00000100:00100000:0.0:1789762659.245856:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014816640 00000100:00100000:0.0:1789762659.245882:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 70 00000100:00100000:2.0:1789762659.245891:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 71 00000100:00100000:0.0:1789762659.245892:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2364:x1876702014816512:12345-192.168.203.48@tcp:2:chmod.500 00000100:00100000:2.0:1789762659.245912:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf566a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702014816640:12345-192.168.203.48@tcp:2:chmod.500 00002000:00010000:0.0:1789762659.245998:0:6881:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1789762659.246002:0:6881:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb mtime from disk: 1789762659 -> 1789762659 00002000:00010000:0.0:1789762659.246004:0:6881:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb atime from disk: 1789762659 -> 1789762659 00002000:00010000:0.0:1789762659.246006:0:6881:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb ctime from disk: 1789762659 -> 1789762659 00002000:00010000:0.0:1789762659.246008:0:6881:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x5:0x0] updating lvb blocks from disk: 1 -> 1 00002000:00010000:2.0:1789762659.246027:0:10916:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x7:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1789762659.246031:0:10916:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x7:0x0] updating lvb mtime from disk: 1789762659 -> 1789762659 00002000:00010000:2.0:1789762659.246033:0:10916:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x7:0x0] updating lvb atime from disk: 1789762659 -> 1789762659 00002000:00010000:2.0:1789762659.246035:0:10916:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x7:0x0] updating lvb ctime from disk: 1789762659 -> 1789762659 00000100:00100000:0.0:1789762659.246038:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf567800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014816512:12345-192.168.203.48@tcp:2:chmod.500 Request processed in 149us (248us total) trans 4294967325 rc 0/0 00002000:00010000:2.0:1789762659.246039:0:10916:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x7:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:0.0:1789762659.246050:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 70 00000100:00100000:2.0:1789762659.246073:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf566a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014816640:12345-192.168.203.48@tcp:2:chmod.500 Request processed in 165us (258us total) trans 4294967431 rc 0/0 00000100:00100000:2.0:1789762659.246080:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 71 00010000:00010000:0.0:1789762659.255160:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762659.268326:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.270896:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014817152 00000100:00100000:2.0:1789762659.270904:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014817152 00000100:00100000:2.0:1789762659.270945:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 113 00000100:00100000:2.0:1789762659.270961:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:11181:x1876702014817152:12345-192.168.203.48@tcp:36:dd.500 00010000:00010000:2.0:1789762659.271008:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762659.271022:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65000/0x416193e012b01e7e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd154 expref: 26 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.271033:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xf:0x0].0x0 (ffff981e82a26900) 00010000:00010000:2.0:1789762659.271036:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762659.271209:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30a00/0x416193e012b01e8c lrc: 3/0,1 mode: --/PW res: [0x200000401:0xf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.271219:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30a00/0x416193e012b01e8c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762659.271226:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30a00/0x416193e012b01e8c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762659.271295:0:6611:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967375 00000004:00010000:2.0:1789762659.271312:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b8590ad3 reply state 000000007c36e2b6 transno 4294967376 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30a00/0x416193e012b01e8c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762659.271319:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762659.271384:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:11181:x1876702014817152:12345-192.168.203.48@tcp:36:dd.500 Request processed in 430us (564us total) trans 4294967376 rc 0/0 00000100:00100000:2.0:1789762659.271390:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 113 00000020:00080000:2.0:1789762659.271394:0:6611:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789762659.272366:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.272451:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014817280 00000100:00100000:0.0:1789762659.272458:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014817280 00000100:00100000:0.0:1789762659.272512:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 72 00000100:00100000:0.0:1789762659.272522:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf566680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:11181:x1876702014817280:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762659.272532:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762659.272542:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b01e93 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd15b expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.272561:0:6649:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b01e23 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd131 expref: 11 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.272569:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762659.272573:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762659.272576:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b01e93 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000080000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd15b expref: 11 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.272599:0:6649:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b01e23 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd131 expref: 11 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.272620:0:6649:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b01e23 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd131 expref: 11 pid: 6649 timeout: 282 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762659.272631:0:6649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d831a9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6649:1876702022128384:192.168.203.48@tcp:104: 00000100:00100000:0.0:1789762659.272691:0:6649:0:(client.c:2715:ptlrpc_set_wait()) set ffff981d82485780 going to sleep for 11 seconds 00000100:00100000:0.0:1789762659.273441:0:6649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d831a9180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6649:1876702022128384:192.168.203.48@tcp:104: 00010000:00010000:0.0:1789762659.273465:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b01e93 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd15b expref: 11 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762659.273485:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8812ce00, rc 0) 00000100:00100000:0.0:1789762659.273513:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf566680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:11181:x1876702014817280:12345-192.168.203.48@tcp:101:dd.500 Request processed in 994us (1151us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762659.273519:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 72 00000100:00100000:2.0:1789762659.273628:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.274930:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.276427:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.277646:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.277806:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.277857:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014817920 00000100:00100000:2.0:1789762659.277867:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014817920 00002000:00010000:2.0:1789762659.277895:0:6652:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1789762659.277904:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d831a8e00 with x1876702014817920 ext(0->18446744073709551615) 00002000:00010000:2.0:1789762659.277916:0:6652:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d831a8e00. 00000100:00100000:2.0:1789762659.277937:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 2 deadline 182781807136 time 182581807136 state 0x1 ref 2 00000100:00100000:2.0:1789762659.277962:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 1 deadline 182781807136 time 182581871846 state 0x0 ref 2 00000100:00100000:2.0:1789762659.277975:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2365:x1876702014817920:12345-192.168.203.48@tcp:10:dd.500 00002000:00010000:2.0:1789762659.278272:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1789762659.278275:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb mtime from disk: 1789762659 -> 1789762659 00002000:00010000:2.0:1789762659.278278:0:6652:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb atime from disk: 1789762659 -> 1789762659 00002000:00010000:2.0:1789762659.278280:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb ctime from disk: 1789762659 -> 1789762659 00002000:00010000:2.0:1789762659.278290:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x4:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1789762659.278332:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:2365:x1876702014817920:12345-192.168.203.48@tcp:10:dd.500 Request processed in 357us (522us total) trans 4294967326 rc 0/0 00000100:00100000:2.0:1789762659.278340:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1 00002000:00010000:2.0:1789762659.278345:0:6652:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/4 (0->18446744073709551615) 00002000:00100000:2.0:1789762659.278347:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d831a8e00 with x1876702014817920 ext(0->18446744073709551615) 00002000:00010000:2.0:1789762659.278352:0:6652:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d831a8e00. 00000100:00100000:2.0:1789762659.288662:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.298476:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.298531:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014818176 00000100:00100000:2.0:1789762659.298538:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014818176 00002000:00100000:2.0:1789762659.298565:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x6:0x0] (1048576->2097151) req@ffff981d831a9f80 x1876702014818176/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:440/0 lens 488/0 e 0 to 0 dl 1789762670 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762659.298573:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d831a9f80 with x1876702014818176 ext(1048576->2097151) 00002000:00010000:2.0:1789762659.298576:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d831a9f80 00000100:00100000:2.0:1789762659.298583:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 182781871846 time 182592711231 state 0x0 ref 1 00000100:00100000:2.0:1789762659.298590:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 182792711231 time 182592711231 state 0x1 ref 1 00000100:00100000:2.0:1789762659.305050:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.305115:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014818304 00000100:00100000:0.0:1789762659.305121:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014818304 00002000:00100000:0.0:1789762659.305143:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x6:0x0] (2097152->3145727) req@ffff981ebf564000 x1876702014818304/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:440/0 lens 488/0 e 0 to 0 dl 1789762670 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762659.305151:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebf564000 with x1876702014818304 ext(2097152->3145727) 00002000:00010000:0.0:1789762659.305154:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebf564000 00000100:00100000:0.0:1789762659.305161:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 182792711231 time 182592711231 state 0x1 ref 2 00000100:00100000:2.0:1789762659.313698:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.321818:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.330990:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.426432:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762659.426488:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762659.426514:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014818944 00000100:00100000:0.0:1789762659.426530:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014818944 00000100:00100000:0.0:1789762659.426633:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 99 00000100:00100000:0.0:1789762659.426662:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014818944:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762659.426756:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014818944:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 103us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762659.426785:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 99 00000100:00100000:2.0:1789762659.959484:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022129280:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.959585:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.959680:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022129792:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.959837:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022129152 00000100:00100000:2.0:1789762659.959849:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022129152 00000100:00100000:2.0:1789762659.959972:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 46 00000100:00100000:2.0:1789762659.960025:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022129152:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762659.960055:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 44236800 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762659.960179:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022129152:12345-0@lo:13:osp-pre-1-0.0 Request processed in 192us (638us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762659.960193:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 46 00000100:00080000:2.0:1789762659.960267:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762659.960279:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762659.960311:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022129408:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.960375:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762659.960514:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022129536 00000100:00100000:2.0:1789762659.960529:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022129536 00000100:00100000:0.0:1789762659.960544:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022129408 00000100:00100000:0.0:1789762659.960576:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022129408 00000100:00100000:2.0:1789762659.960599:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022129792 00000100:00100000:2.0:1789762659.960608:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022129792 00000100:00100000:2.0:1789762659.960685:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:0.0:1789762659.960696:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 102 00000100:00100000:2.0:1789762659.960706:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022129536:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762659.960740:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022129408:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.960840:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022129536:12345-0@lo:400:kworker.0 Request processed in 136us (1012us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762659.960858:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000100:00100000:0.0:1789762659.960864:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022129408:12345-0@lo:400:kworker.0 Request processed in 148us (484us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762659.960880:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 102 00000100:00100000:2.0:1789762659.960950:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022129408:0@lo:400:kworker.0 00000100:00100000:2.0:1789762659.961101:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022129536:0@lo:400:kworker.0 00000001:00080000:0.0:1789762659.977916:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967373 is committed 00000001:00080000:0.0:1789762659.977944:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967376 is committed 00000100:00100000:2.0:1789762660.102521:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.113519:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.120804:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.129990:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.137714:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.144695:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762660.279193:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762660.296296:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.304438:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.494937:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.494998:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014820096 00000100:00100000:2.0:1789762660.495008:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014820096 00002000:00100000:2.0:1789762660.495041:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x4:0x0] (6291456->7340031) req@ffff981ebc584380 x1876702014820096/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/0 e 0 to 0 dl 1789762671 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762660.495049:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc584380 with x1876702014820096 ext(6291456->7340031) 00002000:00010000:2.0:1789762660.495053:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc584380 00000100:00100000:2.0:1789762660.495063:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 183993524641 time 183793524641 state 0x1 ref 8 00000100:00100000:2.0:1789762660.502078:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.502125:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.502170:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014820224 00000100:00100000:0.0:1789762660.502176:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014820224 00002000:00100000:0.0:1789762660.502200:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x4:0x0] (7340032->8388607) req@ffff981ebfb1c700 x1876702014820224/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/0 e 0 to 0 dl 1789762671 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762660.502208:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1c700 with x1876702014820224 ext(7340032->8388607) 00002000:00010000:0.0:1789762660.502212:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfb1c700 00000100:00100000:0.0:1789762660.502219:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 183993524641 time 183793524641 state 0x1 ref 9 00000100:00100000:0.0:1789762660.538904:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.545958:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.546574:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.546617:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014820992 00000100:00100000:0.0:1789762660.546622:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014820992 00000100:00100000:0.0:1789762660.546650:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 75 00000100:00100000:0.0:1789762660.546657:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:11193:x1876702014820992:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762660.546670:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762660.546677:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002fb94978/0x416193e012b01ecb lrc: 2/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd177 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762660.546690:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762660.546693:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762660.546697:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002fb94978/0x416193e012b01ecb lrc: 3/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd177 expref: 25 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762660.546705:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002fb94978/0x416193e012b01ecb lrc: 3/0,0 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd177 expref: 25 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762660.546714:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8812c800, rc 0) 00000100:00100000:0.0:1789762660.546741:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:11193:x1876702014820992:12345-192.168.203.48@tcp:101:dd.0 Request processed in 83us (166us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762660.546749:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 75 00000100:00100000:0.0:1789762660.547451:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.547498:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014821120 00000100:00100000:2.0:1789762660.547505:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014821120 00010000:00010000:2.0:1789762660.547534:0:5790:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b01e54 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd146 expref: 24 pid: 6648 timeout: 283 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762660.547572:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:2.0:1789762660.547582:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2365:x1876702014821120:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762660.547592:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762660.547608:0:5790:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b01e54 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd146 expref: 24 pid: 6648 timeout: 283 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.547618:0:5790:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b01e54 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd146 expref: 24 pid: 6648 timeout: 283 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.547631:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b01e54 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x5:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd146 expref: 24 pid: 6648 timeout: 283 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.547645:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762660.547648:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762660.547654:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000829abbe0/0x416193e012b01ec4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd170 expref: 23 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.547660:0:5790:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000829abbe0/0x416193e012b01ec4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd170 expref: 23 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.547666:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762660.547668:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762660.547671:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000002fb94978/0x416193e012b01ecb lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd177 expref: 23 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.547677:0:5790:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002fb94978/0x416193e012b01ecb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd177 expref: 23 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.547693:0:5790:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000002fb94978/0x416193e012b01ecb lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 4 type: EXT [1048576->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd177 expref: 23 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762660.547700:0:5790:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022130048:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762660.547745:0:5790:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000829abbe0/0x416193e012b01ec4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd170 expref: 23 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762660.547757:0:5790:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022130176:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762660.547788:0:5790:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e905af900 going to sleep for 11 seconds 00000100:00100000:2.0:1789762660.548844:0:5790:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022130048:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762660.549050:0:5790:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022130176:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762660.549068:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762660.549130:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2365:x1876702014821120:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 1549us (1681us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762660.549139:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:0.0:1789762660.553187:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.554435:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.554479:0:7595:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014821376 00000100:00100000:2.0:1789762660.554484:0:7595:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014821376 00010000:00010000:2.0:1789762660.554501:0:7595:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b01e5b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd14d expref: 24 pid: 6649 timeout: 283 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762660.554527:0:7595:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 42 00000100:00100000:2.0:1789762660.554536:0:7595:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2366:x1876702014821376:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762660.554546:0:7595:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762660.554562:0:7595:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b01e5b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd14d expref: 24 pid: 6649 timeout: 283 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.554576:0:7595:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b01e5b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd14d expref: 24 pid: 6649 timeout: 283 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.554590:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b01e5b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd14d expref: 24 pid: 6649 timeout: 283 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.554599:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762660.554608:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762660.554613:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b01ed2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd17e expref: 23 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.554619:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b01ed2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd17e expref: 23 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762660.554637:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b01ed2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd17e expref: 23 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762660.554644:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:7595:1876702022130432:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762660.554679:0:7595:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e905af900 going to sleep for 11 seconds 00000100:00100000:2.0:1789762660.555531:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:7595:1876702022130432:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762660.555544:0:7595:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762660.555573:0:7595:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2366:x1876702014821376:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 1038us (1138us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762660.555582:0:7595:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 42 00000100:00100000:0.0:1789762660.666754:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.666803:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014821504 00000100:00100000:2.0:1789762660.666814:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014821504 00002000:00100000:2.0:1789762660.666862:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x7:0x0] (1048576->2097151) req@ffff981ebc586680 x1876702014821504/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/0 e 0 to 0 dl 1789762671 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762660.666872:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc586680 with x1876702014821504 ext(1048576->2097151) 00002000:00010000:2.0:1789762660.666877:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc586680 00000100:00100000:2.0:1789762660.666896:0:6652:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 3 deadline 0 time 183970804113 state 0x0 ref 1 00000100:00100000:2.0:1789762660.666912:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 3 deadline 183970904113 time 183970804113 state 0x1 ref 1 00000100:00100000:0.0:1789762660.667047:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 183970904113 time 183970955850 state 0x0 ref 1 00000100:00100000:0.0:1789762660.667060:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2365:x1876702014821504:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762660.671877:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.680389:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.685992:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.686173:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 183984530315 time 183990081326 state 0x0 ref 1 00000100:00100000:2.0:1789762660.686189:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2365:x1876702014821888:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762660.704001:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.704245:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.704297:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 183990181326 time 184008203410 state 0x0 ref 1 00000100:00100000:0.0:1789762660.704299:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.704324:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2363:x1876702014822016:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762660.704462:0:6654:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 4 min 3 max 116 00000100:00100000:0.0:1789762660.704474:0:6654:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_004' 00000100:00100000:2.0:1789762660.705066:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184008458119 time 184008974692 state 0x0 ref 3 00000100:00100000:2.0:1789762660.705114:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2366:x1876702014822272:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762660.707590:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2363:x1876702014822016:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3268us (3587us total) trans 4294967441 rc 0/0 00000100:00100000:2.0:1789762660.707600:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 22 00002000:00100000:2.0:1789762660.707611:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x7:0x0] (2097152->3145727) req@ffff981ebfb1d880 x1876702014822016/t4294967441(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/448 e 0 to 0 dl 1789762671 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762660.707621:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1d880 with x1876702014822016 ext(2097152->3145727) 00002000:00010000:2.0:1789762660.707626:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1d880 00000100:00100000:0.0:1789762660.712193:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.712339:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184009074692 time 184016247587 state 0x0 ref 3 00000100:00100000:0.0:1789762660.712355:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2364:x1876702014822400:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762660.714360:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.714397:0:6652:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 5 min 3 max 116 00000100:00100000:0.0:1789762660.714418:0:6652:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_005' 00000100:00100000:0.0:1789762660.714597:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014822528 00000100:00100000:0.0:1789762660.714603:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014822528 00002000:00100000:0.0:1789762660.714629:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x7:0x0] (4194304->5242879) req@ffff981ebc585500 x1876702014822528/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/0 e 0 to 0 dl 1789762671 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762660.714640:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc585500 with x1876702014822528 ext(4194304->5242879) 00002000:00010000:0.0:1789762660.714645:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc585500 00000100:00100000:0.0:1789762660.714655:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184016347587 time 184016247587 state 0x1 ref 4 00000100:00100000:0.0:1789762660.715126:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184016347587 time 184019034928 state 0x0 ref 4 00000100:00100000:0.0:1789762660.715143:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2366:x1876702014822528:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762660.715835:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2364:x1876702014822400:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3483us (3646us total) trans 4294967442 rc 0/0 00000100:00100000:0.0:1789762660.715846:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 25 00002000:00100000:0.0:1789762660.715855:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x7:0x0] (7340032->8388607) req@ffff981ebc584000 x1876702014822400/t4294967442(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/448 e 0 to 0 dl 1789762671 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762660.715879:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc584000 with x1876702014822400 ext(7340032->8388607) 00002000:00010000:0.0:1789762660.715884:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc584000 00000100:00100000:0.0:1789762660.716301:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.716330:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014822656 00000100:00100000:0.0:1789762660.716335:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014822656 00002000:00100000:0.0:1789762660.716357:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x7:0x0] (3145728->4194303) req@ffff981ebf51e680 x1876702014822656/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/0 e 0 to 0 dl 1789762671 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762660.716367:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebf51e680 with x1876702014822656 ext(3145728->4194303) 00002000:00010000:0.0:1789762660.716372:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebf51e680 00000100:00100000:0.0:1789762660.716417:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184019134928 time 184019034928 state 0x1 ref 4 00000100:00100000:0.0:1789762660.716439:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184019134928 time 184020349600 state 0x0 ref 4 00000100:00100000:0.0:1789762660.716454:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2363:x1876702014822656:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762660.717598:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2366:x1876702014822144:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13134us (13351us total) trans 4294967331 rc 0/0 00000100:00100000:2.0:1789762660.717608:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 23 00002000:00100000:2.0:1789762660.717619:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x5:0x0] (0->1048575) req@ffff981ebc584e00 x1876702014822144/t4294967331(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/448 e 0 to 0 dl 1789762671 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762660.717643:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc584e00 with x1876702014822144 ext(0->1048575) 00002000:00010000:2.0:1789762660.717647:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc584e00 00000100:00100000:2.0:1789762660.718169:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2366:x1876702014822528:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3030us (3812us total) trans 4294967443 rc 0/0 00000100:00100000:2.0:1789762660.718176:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 26 00002000:00100000:2.0:1789762660.718182:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x7:0x0] (4194304->5242879) req@ffff981ebc585500 x1876702014822528/t4294967443(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/448 e 0 to 0 dl 1789762671 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762660.718189:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc585500 with x1876702014822528 ext(4194304->5242879) 00002000:00010000:2.0:1789762660.718193:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc585500 00000100:00100000:0.0:1789762660.724976:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762660.725022:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014822912 00000100:00100000:2.0:1789762660.725028:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014822912 00002000:00100000:2.0:1789762660.725049:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x5:0x0] (6291456->7340031) req@ffff981ebf51d180 x1876702014822912/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/0 e 0 to 0 dl 1789762671 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762660.725057:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebf51d180 with x1876702014822912 ext(6291456->7340031) 00002000:00010000:2.0:1789762660.725060:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebf51d180 00000100:00100000:2.0:1789762660.725066:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184019134928 time 184020349600 state 0x0 ref 1 00000100:00100000:2.0:1789762660.725102:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184020449600 time 184020349600 state 0x1 ref 1 00000100:00100000:2.0:1789762660.725120:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184020449600 time 184029031691 state 0x0 ref 1 00000100:00100000:2.0:1789762660.725129:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2363:x1876702014822912:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762660.727183:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.727206:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762660.727228:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00002000:00100000:0.0:1789762660.727277:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x5:0x0] (4194304->5242879) req@ffff981ebf51f100 x1876702014822784/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/0 e 0 to 0 dl 1789762671 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762660.727289:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebf51f100 with x1876702014822784 ext(4194304->5242879) 00002000:00010000:0.0:1789762660.727294:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebf51f100 00000100:00100000:0.0:1789762660.727328:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184029131691 time 184029031691 state 0x1 ref 3 00000100:00100000:0.0:1789762660.727367:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 1 deadline 184031351150 time 184031276848 state 0x0 ref 4 00000100:00100000:0.0:1789762660.727394:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:2363:x1876702014822784:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762660.728024:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2363:x1876702014822912:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2894us (3047us total) trans 4294967333 rc 0/0 00000100:00100000:2.0:1789762660.728039:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 28 00002000:00100000:2.0:1789762660.728048:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x5:0x0] (6291456->7340031) req@ffff981ebf51d180 x1876702014822912/t4294967333(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/448 e 0 to 0 dl 1789762671 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762660.728055:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebf51d180 with x1876702014822912 ext(6291456->7340031) 00002000:00010000:2.0:1789762660.728059:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebf51d180 00000100:00100000:2.0:1789762660.730363:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2363:x1876702014823040:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2993us (3154us total) trans 4294967334 rc 0/0 00000100:00100000:2.0:1789762660.730371:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 29 00002000:00100000:2.0:1789762660.730378:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x5:0x0] (5242880->6291455) req@ffff981ebf51ce00 x1876702014823040/t4294967334(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/448 e 0 to 0 dl 1789762671 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762660.730386:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebf51ce00 with x1876702014823040 ext(5242880->6291455) 00002000:00010000:2.0:1789762660.730389:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebf51ce00 00000100:00100000:2.0:1789762660.731374:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2363:x1876702014822784:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3984us (4189us total) trans 4294967335 rc 0/0 00000100:00100000:2.0:1789762660.731381:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 30 00002000:00100000:2.0:1789762660.731387:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x5:0x0] (4194304->5242879) req@ffff981ebf51f100 x1876702014822784/t4294967335(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/448 e 0 to 0 dl 1789762671 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762660.731393:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebf51f100 with x1876702014822784 ext(4194304->5242879) 00002000:00010000:2.0:1789762660.731396:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebf51f100 00000100:00100000:0.0:1789762660.732428:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014823296 00000100:00100000:0.0:1789762660.732435:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014823296 00002000:00100000:0.0:1789762660.732461:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x5:0x0] (3145728->4194303) req@ffff981ebfed1f80 x1876702014823296/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/0 e 0 to 0 dl 1789762671 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762660.732472:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfed1f80 with x1876702014823296 ext(3145728->4194303) 00002000:00010000:0.0:1789762660.732479:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfed1f80 00000100:00100000:0.0:1789762660.732511:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 0 deadline 184031434712 time 184031334712 state 0x1 ref 2 00000100:00100000:0.0:1789762660.732534:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184031434712 time 184036445432 state 0x0 ref 2 00000100:00100000:0.0:1789762660.732547:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2365:x1876702014823296:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762660.732636:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2364:x1876702014823168:12345-192.168.203.48@tcp:4:dd.0 Request processed in 5196us (5403us total) trans 4294967336 rc 0/0 00000100:00100000:2.0:1789762660.732643:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 31 00002000:00100000:2.0:1789762660.732652:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0x5:0x0] (7340032->8388607) req@ffff981ebf51ca80 x1876702014823168/t4294967336(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/448 e 0 to 0 dl 1789762671 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762660.732665:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebf51ca80 with x1876702014823168 ext(7340032->8388607) 00002000:00010000:2.0:1789762660.732669:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebf51ca80 00000100:00100000:2.0:1789762660.732721:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014823680 00000100:00100000:2.0:1789762660.732725:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014823680 00002000:00100000:2.0:1789762660.732743:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x5:0x0] (2097152->3145727) req@ffff981ebfed0e00 x1876702014823680/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/0 e 0 to 0 dl 1789762671 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762660.732750:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfed0e00 with x1876702014823680 ext(2097152->3145727) 00002000:00010000:2.0:1789762660.732753:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfed0e00 00000100:00100000:2.0:1789762660.732781:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184036545432 time 184036445432 state 0x1 ref 2 00000100:00100000:2.0:1789762660.732804:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184036545432 time 184036715147 state 0x0 ref 2 00000100:00100000:2.0:1789762660.732817:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702014823680:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762660.739285:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2365:x1876702014823680:12345-192.168.203.48@tcp:4:dd.0 Request processed in 6469us (6591us total) trans 4294967338 rc 0/0 00000100:00100000:2.0:1789762660.739293:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 33 00000100:00100000:2.0:1789762660.739298:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184036545432 time 184036715147 state 0x2 ref 0 00002000:00100000:2.0:1789762660.739306:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x5:0x0] (2097152->3145727) req@ffff981ebfed0e00 x1876702014823680/t4294967338(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/448 e 0 to 0 dl 1789762671 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762660.739313:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfed0e00 with x1876702014823680 ext(2097152->3145727) 00002000:00010000:2.0:1789762660.739317:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfed0e00 00000100:00100000:2.0:1789762661.089858:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 184593761426 time 184393761426 state 0x1 ref 7 00000100:00100000:2.0:1789762661.089893:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2366:x1876702014819456:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762661.289948:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 184793851001 time 184593851001 state 0x1 ref 6 00000100:00100000:0.0:1789762661.289976:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2365:x1876702014819584:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762661.292896:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2365:x1876702014819584:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2932us (1155182us total) trans 4294967447 rc 0/0 00000100:00100000:0.0:1789762661.292904:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 12 00002000:00100000:0.0:1789762661.292914:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x6:0x0] (6291456->7340031) req@ffff981ebc585c00 x1876702014819584/t4294967447(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:441/0 lens 488/448 e 0 to 0 dl 1789762671 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762661.292923:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc585c00 with x1876702014819584 ext(6291456->7340031) 00002000:00010000:0.0:1789762661.292929:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc585c00 00000100:00100000:2.0:1789762661.490045:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 184993944023 time 184793944023 state 0x1 ref 5 00000100:00100000:2.0:1789762661.490074:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2366:x1876702014819712:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:2.0:1789762661.690187:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762661.890204:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 185394107448 time 185194107448 state 0x1 ref 3 00000100:00100000:2.0:1789762661.890233:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2366:x1876702014819968:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762662.341535:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762662.378920:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762662.379039:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762662.384621:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.384645:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.384667:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762662.384703:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014824320 00000100:00100000:0.0:1789762662.384720:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014824320 00000100:00100000:2.0:1789762662.384749:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 78 00000100:00100000:2.0:1789762662.384766:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:11228:x1876702014824576:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762662.384777:0:10916:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.384788:0:10916:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a311d697/0x416193e012b01f03 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x6:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd19a expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.384834:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762662.384838:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762662.384845:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a311d697/0x416193e012b01f03 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd19a expref: 46 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.384891:0:10916:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a311d697/0x416193e012b01f03 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd19a expref: 46 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.384937:0:10916:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac31800, rc 0) 00000100:00100000:2.0:1789762662.384987:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:11228:x1876702014824576:12345-192.168.203.48@tcp:101:dd.0 Request processed in 226us (316us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762662.384996:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 78 00000100:00100000:2.0:1789762662.385238:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762662.385287:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014824704 00000100:00100000:0.0:1789762662.385293:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014824704 00000100:00100000:0.0:1789762662.385328:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 80 00000100:00100000:0.0:1789762662.385339:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:11232:x1876702014824704:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762662.385348:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762662.385358:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b01f11 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x6:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd1a1 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.385375:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762662.385379:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x6:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762662.385386:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b01f11 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1a1 expref: 45 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.385414:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b01f11 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1a1 expref: 45 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.385424:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8812de00, rc 0) 00000100:00100000:0.0:1789762662.385460:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+45:11232:x1876702014824704:12345-192.168.203.48@tcp:101:dd.0 Request processed in 123us (220us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762662.385469:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 80 00000100:00100000:0.0:1789762662.386362:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.389036:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762662.389082:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014824960 00000100:00100000:0.0:1789762662.389103:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014824960 00002000:00100000:0.0:1789762662.389128:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x6:0x0] (0->1048575) req@ffff981ebfb1e680 x1876702014824960/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:443/0 lens 488/0 e 0 to 0 dl 1789762673 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762662.389137:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1e680 with x1876702014824960 ext(0->1048575) 00002000:00010000:0.0:1789762662.389146:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1e680 00000100:00100000:0.0:1789762662.389155:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184036545432 time 184036715147 state 0x0 ref 1 00000100:00100000:0.0:1789762662.389163:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184036815147 time 184036715147 state 0x1 ref 1 00000100:00100000:0.0:1789762662.389182:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184036815147 time 185693092792 state 0x0 ref 1 00000100:00100000:0.0:1789762662.389190:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+46:2364:x1876702014824960:12345-192.168.203.48@tcp:3:dd.0 00000100:00100000:0.0:1789762662.393020:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+46:2364:x1876702014824960:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3829us (3983us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762662.393028:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 34 00000100:00100000:0.0:1789762662.393032:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebfa66c00 key gid={0} rule@0xffff981eb5967500 gen 0 rate 10000 depth 3 token 2 deadline 184036815147 time 185693092792 state 0x2 ref 0 00002000:00100000:0.0:1789762662.393041:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x6:0x0] (0->1048575) req@ffff981ebfb1e680 x1876702014824960/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:443/0 lens 488/440 e 0 to 0 dl 1789762673 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762662.393048:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1e680 with x1876702014824960 ext(0->1048575) 00002000:00010000:0.0:1789762662.393052:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1e680 00000100:00100000:0.0:1789762662.431667:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014825216 00000100:00100000:0.0:1789762662.431688:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014825216 00000100:00100000:0.0:1789762662.431748:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 30 00000100:00100000:0.0:1789762662.431760:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc426a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:11242:x1876702014825216:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:0.0:1789762662.431980:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc426a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:11242:x1876702014825216:12345-192.168.203.48@tcp:35:dd.0 Request processed in 223us (396us total) trans 4294967383 rc 0/0 00000100:00100000:0.0:1789762662.431986:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 30 00000100:00100000:2.0:1789762662.442727:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014825344 00000100:00100000:2.0:1789762662.442734:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014825344 00000100:00100000:2.0:1789762662.442768:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 116 00000100:00100000:2.0:1789762662.442774:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:11244:x1876702014825344:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:2.0:1789762662.442787:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.442803:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01f1f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd1af expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.442848:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01f26 lrc: 3/1,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.442859:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01f26 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.442868:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01f26 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762662.442890:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd:0x0]: returning size 16384 00000004:00200000:2.0:1789762662.442893:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762662.442903:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01f26 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.442949:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01f1f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1af expref: 20 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.442979:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01f26 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0xd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1af expref: 19 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.442988:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff981e82a26a00) 00010000:00010000:2.0:1789762662.442991:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762662.442993:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac30400, rc 0) 00000100:00100000:2.0:1789762662.443025:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:11244:x1876702014825344:12345-192.168.203.48@tcp:101:rm.0 Request processed in 251us (375us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762662.443030:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 116 00000100:00100000:2.0:1789762662.444229:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.449343:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762662.449410:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014825984 00000100:00100000:0.0:1789762662.449418:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014825984 00000100:00100000:0.0:1789762662.449463:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 32 00000100:00100000:0.0:1789762662.449472:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:11244:x1876702014825984:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:0.0:1789762662.449586:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:11244:x1876702014825984:12345-192.168.203.48@tcp:35:rm.0 Request processed in 115us (243us total) trans 4294967385 rc 0/0 00000100:00100000:0.0:1789762662.449592:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 32 00000100:00100000:2.0:1789762662.451514:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014826368 00000100:00100000:2.0:1789762662.451521:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014826368 00000100:00100000:2.0:1789762662.451554:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 119 00000100:00100000:2.0:1789762662.451565:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:11244:x1876702014826368:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:2.0:1789762662.451597:0:6611:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7719616512 avail 7713325056 00000100:00100000:2.0:1789762662.451630:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:11244:x1876702014826368:12345-192.168.203.48@tcp:41:rm.0 Request processed in 69us (163us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762662.451636:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 119 00000100:00100000:2.0:1789762662.452425:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762662.454047:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762662.454059:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522200/0x416193e012b01f65 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.454077:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e88522200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762662.454110:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522200/0x416193e012b01f65 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xf:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.454148:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522200/0x416193e012b01f65 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xf:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.454174:0:5793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xf:0x0].0x0 (ffff981e83a46200) 00010000:00010000:0.0:1789762662.454180:0:5793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762662.454204:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522200/0x416193e012b01f65 lrc: 1/0,0 mode: --/PW res: [0x200000401:0xf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.454231:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522200/0x416193e012b01f65 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xf:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.454280:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762662.454282:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762662.454284:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762662.454990:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e88523a00/0x416193e012b01f57 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.454997:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e88523a00/0x416193e012b01f57 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.455008:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e88523a00/0x416193e012b01f57 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.455012:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e88523a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762662.455015:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e88523a00/0x416193e012b01f57 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.455021:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e88523a00/0x416193e012b01f57 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.455028:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x77289514 (ffff981e83a46500) 00010000:00010000:0.0:1789762662.455030:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762662.455033:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e88523a00/0x416193e012b01f57 lrc: 2/0,0 mode: --/PW res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.455037:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e88523a00/0x416193e012b01f57 lrc: 0/0,0 mode: --/PW res: [0x200000401:0xd:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762662.456661:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762662.457431:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762662.457441:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b01f81 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.457459:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762662.457468:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b01f81 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.457481:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b01f81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x10:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.457505:0:8466:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x10:0x0].0x0 (ffff981ec2179800) 00010000:00010000:0.0:1789762662.457518:0:8466:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762662.457528:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b01f81 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x10:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.457541:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b01f81 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x10:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.457575:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762662.457580:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762662.457583:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762662.458887:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.459762:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014827392 00000100:00100000:2.0:1789762662.459774:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014827392 00000100:00100000:2.0:1789762662.459805:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 122 00000100:00100000:2.0:1789762662.459814:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:11244:x1876702014827392:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:2.0:1789762662.459841:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762662.459856:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b01f42 lrc: 0/0,0 mode: --/CR res: [0x200000401:0xd:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1bd expref: 21 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.459872:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b01d74 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0f9 expref: 20 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.459880:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff981e82a26a00) 00010000:00010000:2.0:1789762662.459889:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa65a00/0x416193e012b01d12 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd0e4 expref: 19 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.459895:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:2.0:1789762662.459900:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762662.459911:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01f88 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.459917:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01f88 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.459923:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01f88 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.459930:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01f8f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.459935:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01f8f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.459949:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01f8f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460053:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f96 lrc: 3/0,1 mode: --/EX res: [0x200000401:0xd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460059:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f96 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460064:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f96 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762662.460425:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0xd:0x0]: returning size 16384 00010000:00010000:2.0:1789762662.460433:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f96 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0xd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460438:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f96 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0xd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460442:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f96 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460445:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.460447:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f96 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0xd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460451:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f96 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0xd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460457:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff981e82a26000) 00010000:00010000:2.0:1789762662.460459:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762662.460466:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f96 lrc: 2/0,0 mode: --/EX res: [0x200000401:0xd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460470:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f96 lrc: 0/0,0 mode: --/EX res: [0x200000401:0xd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460482:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01f88 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460486:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01f88 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460491:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01f88 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460494:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.460495:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01f88 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460499:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01f88 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460505:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26e00) 00010000:00010000:2.0:1789762662.460507:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.460510:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01f88 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.460516:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01f88 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762662.460525:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009ee70789 reply state 0000000035b920d8 transno 4294967390 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01f8f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762662.460544:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762662.460587:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:11244:x1876702014827392:12345-192.168.203.48@tcp:36:rm.0 Request processed in 775us (896us total) trans 4294967390 rc 0/0 00000100:00100000:2.0:1789762662.460592:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 122 00010000:00010000:2.0:1789762662.461076:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01f8f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.461107:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01f8f lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.461113:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01f8f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.461118:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.461121:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01f8f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.461125:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01f8f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.461133:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981e82a26b00) 00010000:00010000:2.0:1789762662.461135:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.461137:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01f8f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.461141:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01f8f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762662.746688:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014827520 00000100:00100000:2.0:1789762662.746702:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014827520 00000100:00100000:2.0:1789762662.746782:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 123 00000100:00100000:2.0:1789762662.746793:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e1b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:11255:x1876702014827520:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:2.0:1789762662.746815:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.746833:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f9d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd1c4 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.746888:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01fa4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.746899:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01fa4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.746912:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01fa4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762662.746941:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762662.746944:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762662.746958:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01fa4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.746977:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01f9d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1c4 expref: 21 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.746999:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31a00/0x416193e012b01fa4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1c4 expref: 20 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.747006:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26e00) 00010000:00010000:2.0:1789762662.747008:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762662.747012:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac31a00, rc 0) 00000100:00100000:2.0:1789762662.747074:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e1b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:11255:x1876702014827520:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 284us (444us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762662.747135:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 123 00000100:00100000:2.0:1789762662.754748:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014827904 00000100:00100000:2.0:1789762662.754757:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014827904 00000100:00100000:2.0:1789762662.754795:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 125 00000100:00100000:2.0:1789762662.754804:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:11256:x1876702014827904:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:2.0:1789762662.754817:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.754828:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01fb9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd1cb expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.754865:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01fc0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.754874:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01fc0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.754881:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01fc0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762662.754899:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762662.754901:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762662.754912:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01fc0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.754926:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b01fb9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1cb expref: 22 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.754937:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01fc0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1cb expref: 21 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.754953:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26000) 00010000:00010000:2.0:1789762662.754955:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762662.754957:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac30c00, rc 0) 00000100:00100000:2.0:1789762662.754997:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:11256:x1876702014827904:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 194us (346us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762662.755002:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 125 00000100:00100000:2.0:1789762662.756029:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014828160 00000100:00100000:2.0:1789762662.756034:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014828160 00000100:00100000:2.0:1789762662.756066:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 126 00000100:00100000:2.0:1789762662.756074:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:11256:x1876702014828160:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:2.0:1789762662.756082:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.756084:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1789762662.756129:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b01fc0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1cb expref: 21 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756148:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26000) 00010000:00010000:2.0:1789762662.756167:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762662.756177:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30e00/0x416193e012b01fc7 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd1d2 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756270:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01fce lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756277:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01fce lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756283:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01fce lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756302:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01fd5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756308:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01fd5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756313:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01fd5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756361:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01fdc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756367:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01fdc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756371:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01fdc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762662.756377:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 16384 00010000:00010000:2.0:1789762662.756385:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01fdc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756404:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01fdc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756410:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01fdc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756414:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.756417:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01fdc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756432:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01fdc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756439:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26d00) 00010000:00010000:2.0:1789762662.756441:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.756444:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01fdc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756448:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b01fdc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756463:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01fce lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756467:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01fce lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756472:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01fce lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756476:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.756479:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01fce lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756483:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01fce lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756489:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26a00) 00010000:00010000:2.0:1789762662.756491:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.756493:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01fce lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756505:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b01fce lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756516:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01fd5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756520:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01fd5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756524:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01fd5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756528:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.756531:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01fd5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756535:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01fd5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756540:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981e82a26200) 00010000:00010000:2.0:1789762662.756541:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.756543:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01fd5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756547:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b01fd5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756570:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30e00/0x416193e012b01fc7 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1d2 expref: 21 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756586:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26a00) 00010000:00010000:2.0:1789762662.756588:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.756591:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30e00/0x416193e012b01fc7 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1d2 expref: 21 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.756602:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac30e00, rc 0) 00000100:00100000:2.0:1789762662.756634:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e1aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:11256:x1876702014828160:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 563us (643us total) trans 0 rc 301/301 00000100:00100000:2.0:1789762662.756640:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 126 00000100:00100000:2.0:1789762662.757560:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014828288 00000100:00100000:2.0:1789762662.757565:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014828288 00000100:00100000:2.0:1789762662.757589:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 127 00000100:00100000:2.0:1789762662.757596:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e1a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:11256:x1876702014828288:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:2.0:1789762662.757604:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.757615:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b01fe3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd1d9 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.757641:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31000/0x416193e012b01fea lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.757648:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31000/0x416193e012b01fea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.757654:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31000/0x416193e012b01fea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762662.757667:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762662.757675:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762662.757685:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31000/0x416193e012b01fea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.757699:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b01fe3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1d9 expref: 22 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.757709:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31000/0x416193e012b01fea lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1d9 expref: 21 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.757715:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762662.757717:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762662.757719:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac31000, rc 0) 00000100:00100000:2.0:1789762662.757750:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e1a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:11256:x1876702014828288:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 156us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762662.757756:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 127 00000100:00100000:2.0:1789762662.758588:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014828544 00000100:00100000:2.0:1789762662.758593:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014828544 00000100:00100000:2.0:1789762662.758620:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 128 00000100:00100000:2.0:1789762662.758626:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:11256:x1876702014828544:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:2.0:1789762662.758633:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.758641:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b01ff1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd1e0 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758679:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b01ff8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758686:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b01ff8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758699:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b01ff8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758706:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b01fff lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758711:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b01fff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758716:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b01fff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762662.758751:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 16384 00010000:00010000:2.0:1789762662.758816:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b01ff8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758821:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b01ff8 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758825:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b01ff8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758829:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.758832:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b01ff8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758836:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b01ff8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758843:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762662.758845:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.758853:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b01ff8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758858:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b01ff8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758867:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b01fff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758871:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b01fff lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758875:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b01fff lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758879:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.758880:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b01fff lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758885:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b01fff lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758889:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981e82a26600) 00010000:00010000:2.0:1789762662.758890:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.758892:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b01fff lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758896:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b01fff lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758916:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b01ff1 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1e0 expref: 23 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758932:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762662.758934:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762662.758937:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b01ff1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1e0 expref: 23 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.758947:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac31200, rc 0) 00000100:00100000:2.0:1789762662.758982:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:11256:x1876702014828544:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 357us (431us total) trans 4294967392 rc 301/301 00000100:00100000:2.0:1789762662.758988:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 128 00000100:00100000:2.0:1789762662.759767:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014828800 00000100:00100000:2.0:1789762662.759772:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014828800 00000100:00100000:2.0:1789762662.759800:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 129 00000100:00100000:2.0:1789762662.759809:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e19180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:11256:x1876702014828800:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:2.0:1789762662.759839:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30000/0x416193e012b0200d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.759846:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30000/0x416193e012b0200d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.759853:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30000/0x416193e012b0200d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762662.759980:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cf42de41 reply state 00000000daea325f transno 4294967393 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30000/0x416193e012b0200d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762662.759988:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762662.759993:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 16384 00000100:00100000:2.0:1789762662.760044:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e19180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:11256:x1876702014828800:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 237us (316us total) trans 4294967393 rc 0/0 00000100:00100000:2.0:1789762662.760051:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 129 00010000:00010000:0.0:1789762662.760519:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30000/0x416193e012b0200d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.760528:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30000/0x416193e012b0200d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.760534:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30000/0x416193e012b0200d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.760538:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762662.760541:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30000/0x416193e012b0200d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.760545:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30000/0x416193e012b0200d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.760553:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26100) 00010000:00010000:0.0:1789762662.760555:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789762662.760558:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30000/0x416193e012b0200d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.760562:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30000/0x416193e012b0200d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762662.760760:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014829056 00000100:00100000:0.0:1789762662.760765:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014829056 00000100:00100000:0.0:1789762662.760797:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 34 00000100:00100000:0.0:1789762662.760808:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b1b450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:11256:x1876702014829056:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:0.0:1789762662.760916:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b1b450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:11256:x1876702014829056:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 112us (192us total) trans 4294967394 rc 0/0 00000100:00100000:0.0:1789762662.760925:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 34 00000100:00100000:2.0:1789762662.764370:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014829184 00000100:00100000:2.0:1789762662.764376:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014829184 00000100:00100000:2.0:1789762662.764418:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 130 00000100:00100000:2.0:1789762662.764427:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e1b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:11257:x1876702014829184:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:2.0:1789762662.764437:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.764445:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02014 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd1e7 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764471:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b0201b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764480:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b0201b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764486:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b0201b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764494:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02022 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764500:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02022 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764505:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02022 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764558:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b02029 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764579:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b02029 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764585:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b02029 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762662.764600:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 16384 00000004:00200000:2.0:1789762662.764602:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762662.764612:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b02029 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764619:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b0201b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764623:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b0201b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764627:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b0201b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764631:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.764634:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b0201b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764638:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b0201b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764644:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762662.764646:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.764649:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b0201b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764661:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b0201b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764670:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02022 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764674:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02022 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764678:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02022 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764682:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.764684:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02022 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764688:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02022 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764693:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981e82a26600) 00010000:00010000:2.0:1789762662.764695:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.764697:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02022 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764701:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02022 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764719:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02014 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1e7 expref: 26 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764727:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b02029 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1e7 expref: 25 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.764738:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26a00) 00010000:00010000:2.0:1789762662.764740:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762662.764742:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac30600, rc 0) 00000100:00100000:2.0:1789762662.764775:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e1b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:11257:x1876702014829184:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 351us (465us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762662.764781:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 130 00000100:00100000:2.0:1789762662.765682:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014829440 00000100:00100000:2.0:1789762662.765690:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014829440 00000100:00100000:2.0:1789762662.765731:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 131 00000100:00100000:2.0:1789762662.765747:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:11257:x1876702014829440:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:2.0:1789762662.765773:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762662.765788:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b02029 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1e7 expref: 25 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.765804:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26a00) 00010000:00010000:2.0:1789762662.765812:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762662.765826:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30e00/0x416193e012b02030 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.765836:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30e00/0x416193e012b02030 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.765846:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30e00/0x416193e012b02030 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762662.765994:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fafc8919 reply state 00000000eb413dfb transno 4294967395 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30e00/0x416193e012b02030 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762662.766017:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762662.766023:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 16384 00000100:00100000:2.0:1789762662.766127:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:11257:x1876702014829440:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 346us (455us total) trans 4294967395 rc 0/0 00000100:00100000:2.0:1789762662.766139:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 131 00000100:00100000:2.0:1789762662.770917:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014829568 00000100:00100000:2.0:1789762662.770925:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014829568 00000100:00100000:2.0:1789762662.770972:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 132 00000100:00100000:2.0:1789762662.770985:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:11258:x1876702014829568:12345-192.168.203.48@tcp:101:touch.500 00010000:00010000:2.0:1789762662.770999:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.771013:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b02037 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd1ee expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771052:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b0203e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771062:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b0203e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771073:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b0203e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771130:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02045 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771141:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02045 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771151:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02045 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771227:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b0204c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771237:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b0204c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771245:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b0204c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762662.771262:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 16384 00000004:00200000:2.0:1789762662.771266:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762662.771280:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b0204c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771289:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b0203e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771296:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b0203e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771303:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b0203e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771309:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.771314:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b0203e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771321:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b0203e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771332:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762662.771344:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.771349:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b0203e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771356:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30400/0x416193e012b0203e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771372:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02045 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771379:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02045 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771401:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02045 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771408:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.771412:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02045 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771419:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02045 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771429:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981e82a26d00) 00010000:00010000:2.0:1789762662.771432:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.771436:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02045 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771442:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02045 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771471:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31400/0x416193e012b02037 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1ee expref: 27 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771502:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b0204c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1ee expref: 26 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.771509:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26000) 00010000:00010000:2.0:1789762662.771512:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789762662.771515:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac30c00, rc 0) 00000100:00100000:2.0:1789762662.771557:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:11258:x1876702014829568:12345-192.168.203.48@tcp:101:touch.500 Request processed in 575us (693us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762662.771566:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 132 00000100:00100000:2.0:1789762662.772480:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014829824 00000100:00100000:2.0:1789762662.772487:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014829824 00000100:00100000:2.0:1789762662.772527:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 133 00000100:00100000:2.0:1789762662.772541:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e18e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:11258:x1876702014829824:12345-192.168.203.48@tcp:101:touch.500 00010000:00010000:2.0:1789762662.772550:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.772559:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b02053 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd1f5 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.772628:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b0205a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.772640:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b0205a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.772650:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b0205a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.772668:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02061 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.772685:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02061 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.772694:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02061 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762662.773293:0:5804:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x8:0x0], index 0: 0 00010000:00010000:2.0:1789762662.773527:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b02068 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x15:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.773540:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b02068 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.773550:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b02068 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.773580:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b0205a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.773588:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b0205a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.773595:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b0205a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.773602:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.773606:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b0205a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.773614:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b0205a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.773639:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26000) 00010000:00010000:2.0:1789762662.773643:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.773647:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b0205a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.773654:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b0205a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762662.773668:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000067bfbf18 reply state 000000003497e0a1 transno 4294967396 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02061 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762662.773677:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762662.773705:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b02053 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1f5 expref: 29 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.773721:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31c00/0x416193e012b02068 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1f5 expref: 28 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.773730:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x15:0x0].0x0 (ffff981e82a26600) 00010000:00010000:2.0:1789762662.773733:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762662.773736:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac31c00, rc 0) 00000100:00100000:2.0:1789762662.773809:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e18e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:11258:x1876702014829824:12345-192.168.203.48@tcp:101:touch.500 Request processed in 1273us (1374us total) trans 4294967396 rc 0/0 00000100:00100000:2.0:1789762662.773819:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 133 00010000:00010000:2.0:1789762662.774923:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02061 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.774933:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02061 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.774945:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02061 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.774950:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.774953:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02061 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.774958:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02061 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.774968:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x4009f790 (ffff981e82a26a00) 00010000:00010000:2.0:1789762662.774971:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.774974:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02061 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.774978:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02061 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.777036:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522c00/0x416193e012b02076 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.777044:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522c00/0x416193e012b02076 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.777049:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522c00/0x416193e012b02076 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.777053:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e88522c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.777056:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522c00/0x416193e012b02076 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.777072:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522c00/0x416193e012b02076 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.777082:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x15:0x0].0x0 (ffff981e83a46200) 00010000:00010000:2.0:1789762662.777184:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.777188:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522c00/0x416193e012b02076 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x15:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.777194:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522c00/0x416193e012b02076 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x15:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762662.777317:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.777362:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014830208 00000100:00100000:2.0:1789762662.777369:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014830208 00000100:00100000:2.0:1789762662.777431:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 135 00000100:00100000:2.0:1789762662.777446:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:11258:x1876702014830208:12345-192.168.203.48@tcp:101:touch.500 00010000:00010000:2.0:1789762662.777467:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.777488:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b0207d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x15:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd1fc expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.777533:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b0207d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1fc expref: 29 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.777546:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b0207d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1fc expref: 29 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.777560:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x15:0x0].0x0 (ffff981e82a26400) 00010000:00010000:2.0:1789762662.777563:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789762662.777571:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac31600, rc 0) 00000100:00100000:2.0:1789762662.777606:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:11258:x1876702014830208:12345-192.168.203.48@tcp:101:touch.500 Request processed in 167us (296us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762662.777611:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 135 00000100:00100000:2.0:1789762662.778850:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.778893:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014830336 00000100:00100000:2.0:1789762662.778898:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014830336 00000100:00100000:2.0:1789762662.778942:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 82 00000100:00100000:2.0:1789762662.778951:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+46:2364:x1876702014830336:12345-192.168.203.48@tcp:2:touch.500 00000100:00100000:2.0:1789762662.779694:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+47:2364:x1876702014830336:12345-192.168.203.48@tcp:2:touch.500 Request processed in 742us (841us total) trans 4294967449 rc 0/0 00000100:00100000:2.0:1789762662.779709:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 82 00000100:00100000:2.0:1789762662.780566:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.785080:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014830720 00000100:00100000:2.0:1789762662.785108:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014830720 00000100:00100000:2.0:1789762662.785157:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 136 00000100:00100000:2.0:1789762662.785169:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:11259:x1876702014830720:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:2.0:1789762662.785180:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762662.785189:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02084 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd203 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785232:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b0208b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785240:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b0208b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785247:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b0208b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785263:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02092 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x13:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785268:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02092 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785273:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02092 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785332:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02099 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785347:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02099 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x15:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785352:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02099 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x15:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762662.785373:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x15:0x0]: returning size 0 00000004:00200000:2.0:1789762662.785376:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762662.785400:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02099 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x15:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785410:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02099 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785416:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b0208b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785421:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b0208b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785435:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b0208b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785440:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.785443:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b0208b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785449:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b0208b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785456:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26000) 00010000:00010000:2.0:1789762662.785459:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.785461:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b0208b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785465:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30600/0x416193e012b0208b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785474:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02092 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785478:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02092 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785484:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02092 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785490:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762662.785493:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02092 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785513:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02092 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785524:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x4009f790 (ffff981e82a26a00) 00010000:00010000:2.0:1789762662.785526:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762662.785530:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02092 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785537:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31200/0x416193e012b02092 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785571:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b02084 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd203 expref: 32 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785582:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02099 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd203 expref: 31 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.785587:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x15:0x0].0x0 (ffff981e82a26400) 00010000:00010000:2.0:1789762662.785589:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762662.785591:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac31e00, rc 0) 00000100:00100000:2.0:1789762662.785627:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:11259:x1876702014830720:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 463us (586us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762662.785633:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 136 00000100:00100000:2.0:1789762662.786634:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.788054:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014831104 00000100:00100000:2.0:1789762662.788062:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014831104 00000100:00100000:2.0:1789762662.788137:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 137 00000100:00100000:2.0:1789762662.788149:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:11259:x1876702014831104:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:2.0:1789762662.788182:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762662.788201:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31e00/0x416193e012b02099 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd203 expref: 31 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.788221:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x15:0x0].0x0 (ffff981e82a26400) 00010000:00010000:2.0:1789762662.788225:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762662.788243:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30800/0x416193e012b020a7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.788253:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30800/0x416193e012b020a7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.788261:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30800/0x416193e012b020a7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762662.788438:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000061d211bb reply state 00000000e18858fc transno 4294967399 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30800/0x416193e012b020a7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762662.788449:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762662.788522:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:11259:x1876702014831104:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 375us (510us total) trans 4294967399 rc 0/0 00000100:00100000:2.0:1789762662.788530:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 137 00010000:00010000:0.0:1789762662.789742:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30800/0x416193e012b020a7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.789757:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30800/0x416193e012b020a7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.789767:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30800/0x416193e012b020a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.789774:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762662.789780:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30800/0x416193e012b020a7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.789789:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30800/0x416193e012b020a7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.789804:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x15:0x0].0x0 (ffff981e82a26400) 00010000:00010000:0.0:1789762662.789808:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762662.789812:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30800/0x416193e012b020a7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.789820:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30800/0x416193e012b020a7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762662.790047:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014831232 00000100:00100000:2.0:1789762662.790054:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014831232 00000100:00100000:2.0:1789762662.790113:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 84 00000100:00100000:2.0:1789762662.790127:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+48:2364:x1876702014831232:12345-192.168.203.48@tcp:2:chmod.0 00002000:00010000:2.0:1789762662.790271:0:6647:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1789762662.790274:0:6647:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb mtime from disk: 1789762662 -> 1789762662 00002000:00010000:2.0:1789762662.790276:0:6647:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb atime from disk: 1789762662 -> 1789762662 00002000:00010000:2.0:1789762662.790283:0:6647:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb ctime from disk: 1789762662 -> 1789762662 00002000:00010000:2.0:1789762662.790285:0:6647:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1789762662.790322:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+49:2364:x1876702014831232:12345-192.168.203.48@tcp:2:chmod.0 Request processed in 203us (330us total) trans 4294967450 rc 0/0 00000100:00100000:2.0:1789762662.790328:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 84 00000100:00100000:2.0:1789762662.809891:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.811875:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014831744 00000100:00100000:2.0:1789762662.811884:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014831744 00000100:00100000:2.0:1789762662.811926:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 139 00000100:00100000:2.0:1789762662.811934:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:11265:x1876702014831744:12345-192.168.203.48@tcp:36:dd.500 00010000:00010000:2.0:1789762662.811985:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762662.812004:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e88522600/0x416193e012b020c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd211 expref: 33 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.812020:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x15:0x0].0x0 (ffff981e82a26400) 00010000:00010000:2.0:1789762662.812023:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762662.812191:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b020d1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.812202:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b020d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.812209:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b020d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762662.812243:0:5804:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967401 00000004:00010000:2.0:1789762662.812266:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008535cca0 reply state 00000000303cd524 transno 4294967402 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30200/0x416193e012b020d1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762662.812282:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762662.812347:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:11265:x1876702014831744:12345-192.168.203.48@tcp:36:dd.500 Request processed in 414us (552us total) trans 4294967402 rc 0/0 00000100:00100000:2.0:1789762662.812355:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 139 00000020:00080000:2.0:1789762662.812360:0:5804:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789762662.813078:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762662.813142:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014831872 00000100:00100000:0.0:1789762662.813148:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014831872 00000100:00100000:0.0:1789762662.813183:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 85 00000100:00100000:0.0:1789762662.813193:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+49:11265:x1876702014831872:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762662.813202:0:10916:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762662.813212:0:10916:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b020d8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd218 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.813233:0:10916:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bbc9a884/0x416193e012b020a0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd20a expref: 50 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.813240:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762662.813244:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762662.813247:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b020d8 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000080000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd218 expref: 50 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.813271:0:10916:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bbc9a884/0x416193e012b020a0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd20a expref: 50 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.813281:0:10916:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bbc9a884/0x416193e012b020a0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd20a expref: 50 pid: 6649 timeout: 286 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762662.813292:0:10916:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebf51e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10916:1876702022130560:192.168.203.48@tcp:104: 00000100:00100000:0.0:1789762662.813357:0:10916:0:(client.c:2715:ptlrpc_set_wait()) set ffff981d82485e40 going to sleep for 11 seconds 00000100:00100000:0.0:1789762662.814040:0:10916:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebf51e300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10916:1876702022130560:192.168.203.48@tcp:104: 00010000:00010000:0.0:1789762662.814066:0:10916:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b020d8 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd218 expref: 50 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762662.814078:0:10916:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8812c400, rc 0) 00000100:00100000:0.0:1789762662.814126:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+50:11265:x1876702014831872:12345-192.168.203.48@tcp:101:dd.500 Request processed in 934us (1047us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762662.814133:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 85 00000100:00100000:2.0:1789762662.814215:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.815323:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762662.819337:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762662.819369:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014832384 00000100:00100000:2.0:1789762662.819374:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014832384 00010000:00010000:2.0:1789762662.819401:0:5791:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b020d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000480020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd218 expref: 51 pid: 10916 timeout: 286 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762662.819426:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 44 00000100:00100000:2.0:1789762662.819433:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+51:2364:x1876702014832384:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762662.819440:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762662.819453:0:5791:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b020d8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000480020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd218 expref: 51 pid: 10916 timeout: 286 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.819461:0:5791:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b020d8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000480020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd218 expref: 51 pid: 10916 timeout: 286 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.819471:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b020d8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801480020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd218 expref: 51 pid: 10916 timeout: 286 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.819493:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789762662.819497:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789762662.819502:0:5791:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b020df lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.819511:0:5791:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b020df lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762662.819533:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762662.819558:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+50:2364:x1876702014832384:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 126us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762662.819566:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 44 00000001:00080000:0.0:1789762662.848957:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967447 is committed 00000001:00080000:0.0:1789762662.848980:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967452 is committed 00000100:00100000:0.0:1789762662.861729:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762663.018604:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186322361113 time 186322484913 state 0x0 ref 1 00000100:00100000:2.0:1789762663.018676:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2366:x1876702014832512:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762663.018755:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 1048576, mtime: 0x6aad9c67, atime: 0x6aad9c66, ctime: 0x6aad9c67, blocks: 0x803 00010000:00010000:2.0:1789762663.018769:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002199f73d/0x416193e012b020e6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.018790:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=186s v=(0 0) 00010000:00010000:2.0:1789762663.018796:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762663.018804:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002199f73d/0x416193e012b020e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.018835:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002199f73d/0x416193e012b020e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762663.035684:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:2.0:1789762663.035693:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 2051 -> 4103 00000001:00080000:0.0:1789762663.035704:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967453 is committed 00010000:00010000:2.0:1789762663.035707:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002199f73d/0x416193e012b020e6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.035726:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002199f73d/0x416193e012b020e6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.035737:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002199f73d/0x416193e012b020e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.035746:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762663.035751:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002199f73d/0x416193e012b020e6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.035761:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002199f73d/0x416193e012b020e6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.035779:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002199f73d/0x416193e012b020e6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.035787:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002199f73d/0x416193e012b020e6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762663.035871:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2366:x1876702014832512:12345-192.168.203.48@tcp:4:dd.500 Request processed in 17246us (174144us total) trans 4294967453 rc 0/0 00000100:00100000:2.0:1789762663.035886:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 37 00000100:00100000:2.0:1789762663.035895:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186322361113 time 186322484913 state 0x2 ref 0 00000100:00100000:0.0:1789762663.038962:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00002000:00010000:0.0:1789762663.232272:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:0.0:1789762663.232279:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:0.0:1789762663.232293:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f656d781/0x416193e012b020ed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.232307:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f656d781/0x416193e012b020ed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.232319:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f656d781/0x416193e012b020ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.232328:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e88523c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762663.232333:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f656d781/0x416193e012b020ed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.232356:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f656d781/0x416193e012b020ed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.232377:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f656d781/0x416193e012b020ed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.232401:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f656d781/0x416193e012b020ed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762663.232483:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014832640:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13796us (193525us total) trans 4294967454 rc 0/0 00000100:00100000:0.0:1789762663.232494:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 38 00000100:00100000:0.0:1789762663.232503:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186522484913 time 186522570364 state 0x2 ref 0 00000100:00100000:2.0:1789762663.234964:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014832768 00000100:00100000:2.0:1789762663.234973:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014832768 00000100:00100000:2.0:1789762663.235015:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186522484913 time 186522570364 state 0x0 ref 1 00000100:00100000:2.0:1789762663.235027:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186722570364 time 186522570364 state 0x1 ref 1 00000100:00100000:2.0:1789762663.418737:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186722570364 time 186722637333 state 0x0 ref 1 00000100:00100000:2.0:1789762663.418773:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014832768:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762663.418829:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 3145728, mtime: 0x6aad9c67, atime: 0x6aad9c66, ctime: 0x6aad9c67, blocks: 0x1807 00010000:00010000:2.0:1789762663.418839:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b020f4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.418855:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=186s v=(0 0) 00010000:00010000:2.0:1789762663.418860:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762663.418875:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b020f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.418893:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b020f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762663.432027:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:2.0:1789762663.432033:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 6151 -> 8199 00000001:00080000:0.0:1789762663.432036:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967455 is committed 00010000:00010000:2.0:1789762663.432046:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b020f4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.432063:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b020f4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.432070:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b020f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.432075:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762663.432078:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b020f4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.432101:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b020f4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.432117:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b020f4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.432122:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b020f4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762663.432191:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014832768:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13426us (197283us total) trans 4294967455 rc 0/0 00000100:00100000:2.0:1789762663.432200:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 39 00000100:00100000:2.0:1789762663.432206:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186722570364 time 186722637333 state 0x2 ref 0 00000100:00100000:0.0:1789762663.434695:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014832896 00000100:00100000:0.0:1789762663.434705:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014832896 00000100:00100000:0.0:1789762663.434750:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186722570364 time 186722637333 state 0x0 ref 1 00000100:00100000:0.0:1789762663.434762:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186922637333 time 186722637333 state 0x1 ref 1 00000100:00100000:0.0:1789762663.618803:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186922637333 time 186922706461 state 0x0 ref 1 00000100:00100000:0.0:1789762663.618830:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014832896:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762663.618880:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 4194304, mtime: 0x6aad9c67, atime: 0x6aad9c66, ctime: 0x6aad9c67, blocks: 0x2007 00010000:00010000:0.0:1789762663.618888:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b020fb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.618914:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=186s v=(0 0) 00010000:00010000:0.0:1789762663.618918:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762663.618923:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b020fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.618934:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b020fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762663.632254:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967456 is committed 00002000:00010000:0.0:1789762663.632290:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:0.0:1789762663.632293:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:0.0:1789762663.632303:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b020fb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.632312:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b020fb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.632318:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b020fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.632323:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762663.632327:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b020fb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.632335:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b020fb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.632369:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b020fb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.632392:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b020fb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762663.632451:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014832896:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13632us (197815us total) trans 4294967456 rc 0/0 00000100:00100000:0.0:1789762663.632460:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 40 00000100:00100000:0.0:1789762663.632465:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186922637333 time 186922706461 state 0x2 ref 0 00000100:00100000:2.0:1789762663.634794:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014833024 00000100:00100000:2.0:1789762663.634804:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014833024 00000100:00100000:2.0:1789762663.634851:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 186922637333 time 186922706461 state 0x0 ref 1 00000100:00100000:2.0:1789762663.634864:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187122706461 time 186922706461 state 0x1 ref 1 00000100:00100000:2.0:1789762663.818875:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187122706461 time 187122777696 state 0x0 ref 1 00000100:00100000:2.0:1789762663.818903:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014833024:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762663.818960:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 5242880, mtime: 0x6aad9c67, atime: 0x6aad9c66, ctime: 0x6aad9c67, blocks: 0x2807 00010000:00010000:2.0:1789762663.818968:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b02102 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.818984:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=187s v=(0 0) 00010000:00010000:2.0:1789762663.818988:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762663.818995:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b02102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762663.819012:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b02102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762663.831253:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:0.0:1789762663.831258:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:0.0:1789762663.831268:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b02102 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.831278:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b02102 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.831284:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b02102 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.831289:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762663.831293:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b02102 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.831309:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b02102 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.831325:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b02102 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762663.831330:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005b6a09c/0x416193e012b02102 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762663.831408:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014833024:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12515us (196671us total) trans 4294967457 rc 0/0 00000100:00100000:0.0:1789762663.831420:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:0.0:1789762663.831427:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187122706461 time 187122777696 state 0x2 ref 0 00000100:00100000:0.0:1789762663.833901:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014833152 00000100:00100000:0.0:1789762663.833910:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014833152 00000100:00100000:0.0:1789762663.833947:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187122706461 time 187122777696 state 0x0 ref 1 00000100:00100000:0.0:1789762663.833956:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187322777696 time 187122777696 state 0x1 ref 1 00000100:00100000:0.0:1789762664.018938:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187322777696 time 187322840556 state 0x0 ref 1 00000100:00100000:0.0:1789762664.018972:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014833152:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762664.019022:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 6291456, mtime: 0x6aad9c67, atime: 0x6aad9c66, ctime: 0x6aad9c67, blocks: 0x3007 00010000:00010000:0.0:1789762664.019029:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02109 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.019059:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=187s v=(0 0) 00010000:00010000:0.0:1789762664.019064:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762664.019072:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.019113:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762664.031762:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:0.0:1789762664.031768:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb mtime from disk: 1789762663 -> 1789762664 00002000:00010000:0.0:1789762664.031771:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb ctime from disk: 1789762663 -> 1789762664 00002000:00010000:0.0:1789762664.031774:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:0.0:1789762664.031788:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02109 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.031801:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02109 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.031813:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02109 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.031821:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762664.031826:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02109 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.031836:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02109 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.031864:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02109 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.031870:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02109 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762664.031941:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014833152:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12985us (198082us total) trans 4294967458 rc 0/0 00000100:00100000:0.0:1789762664.031950:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 42 00000100:00100000:0.0:1789762664.031956:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187322777696 time 187322840556 state 0x2 ref 0 00000100:00100000:2.0:1789762664.034441:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014833280 00000100:00100000:2.0:1789762664.034450:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014833280 00000100:00100000:2.0:1789762664.034489:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187322777696 time 187322840556 state 0x0 ref 1 00000100:00100000:2.0:1789762664.034500:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187522840556 time 187322840556 state 0x1 ref 1 00000100:00100000:0.0:1789762664.219029:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187522840556 time 187522931824 state 0x0 ref 1 00000100:00100000:0.0:1789762664.219057:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014833280:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762664.219137:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 7340032, mtime: 0x6aad9c68, atime: 0x6aad9c66, ctime: 0x6aad9c68, blocks: 0x3807 00010000:00010000:0.0:1789762664.219145:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02110 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.219158:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=187s v=(0 0) 00010000:00010000:0.0:1789762664.219162:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762664.219166:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.219177:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762664.230540:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:0.0:1789762664.230546:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:0.0:1789762664.230556:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02110 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.230565:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02110 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.230571:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02110 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.230576:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762664.230580:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02110 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.230587:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02110 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.230601:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02110 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.230607:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02110 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762664.230662:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014833280:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11617us (196287us total) trans 4294967459 rc 0/0 00000100:00100000:0.0:1789762664.230675:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 43 00000100:00100000:0.0:1789762664.230681:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187522840556 time 187522931824 state 0x2 ref 0 00000100:00100000:0.0:1789762664.233124:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014833408 00000100:00100000:0.0:1789762664.233132:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014833408 00000100:00100000:0.0:1789762664.233164:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187522840556 time 187522931824 state 0x0 ref 1 00000100:00100000:0.0:1789762664.233173:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187722931824 time 187522931824 state 0x1 ref 1 00000100:00100000:0.0:1789762664.419111:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187722931824 time 187722996381 state 0x0 ref 1 00000100:00100000:0.0:1789762664.419135:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2366:x1876702014833408:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762664.419182:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 8388608, mtime: 0x6aad9c68, atime: 0x6aad9c66, ctime: 0x6aad9c68, blocks: 0x4007 00010000:00010000:0.0:1789762664.419188:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02117 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.419202:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=187s v=(0 0) 00010000:00010000:0.0:1789762664.419206:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762664.419211:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.419221:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762664.431720:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967460 is committed 00002000:00010000:0.0:1789762664.431737:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:0.0:1789762664.431742:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:0.0:1789762664.431754:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02117 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.431771:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02117 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.431779:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02117 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.431784:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762664.431787:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02117 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.431794:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02117 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.431808:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02117 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.431813:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02117 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762664.431884:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2366:x1876702014833408:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12755us (198818us total) trans 4294967460 rc 0/0 00000100:00100000:0.0:1789762664.431892:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 44 00000100:00100000:0.0:1789762664.431898:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187722931824 time 187722996381 state 0x2 ref 0 00000100:00100000:2.0:1789762664.434109:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014833536 00000100:00100000:2.0:1789762664.434119:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014833536 00000100:00100000:2.0:1789762664.434156:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187722931824 time 187722996381 state 0x0 ref 1 00000100:00100000:2.0:1789762664.434173:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187922996381 time 187722996381 state 0x1 ref 1 00000100:00100000:0.0:1789762664.546355:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762664.546414:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762664.546421:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014833792 00000100:00100000:2.0:1789762664.546430:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014833792 00000100:00100000:0.0:1789762664.546457:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762664.546494:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762664.546513:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014834048 00000100:00100000:2.0:1789762664.546518:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014834048 00000100:00100000:0.0:1789762664.546536:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014833920 00000100:00100000:0.0:1789762664.546543:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014833920 00000100:00100000:2.0:1789762664.546562:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 104 00000100:00100000:2.0:1789762664.546577:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014834048:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762664.546585:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 54 00000100:00100000:0.0:1789762664.546594:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2366:x1876702014833920:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762664.546642:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014834048:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 65us (148us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762664.546650:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2366:x1876702014833920:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 53us (188us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762664.546667:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 104 00000100:00100000:0.0:1789762664.546671:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 54 00000004:00080000:2.0:1789762664.567342:0:6868:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 187871232256ns, next at 188871226679ns, rc = 0 00000100:00100000:2.0:1789762664.567538:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022130944 00000100:00100000:2.0:1789762664.567546:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022130944 00000100:00100000:2.0:1789762664.567607:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 47 00000100:00100000:2.0:1789762664.567618:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022130944:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762664.567637:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86835200 pending 0 free 3859808256 avail 3831496704 00000100:00100000:2.0:1789762664.567691:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022130944:12345-0@lo:13:osp-pre-0-0.0 Request processed in 74us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762664.567700:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 47 00000100:00100000:0.0:1789762664.619168:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187922996381 time 187923070833 state 0x0 ref 1 00000100:00100000:0.0:1789762664.619191:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2365:x1876702014833536:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762664.619245:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 9437184, mtime: 0x6aad9c68, atime: 0x6aad9c66, ctime: 0x6aad9c68, blocks: 0x4807 00010000:00010000:0.0:1789762664.619256:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b0211e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.619274:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=187s v=(0 0) 00010000:00010000:0.0:1789762664.619279:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762664.619286:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b0211e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.619300:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b0211e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762664.630063:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:2.0:1789762664.630070:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 18439 -> 20487 00010000:00010000:2.0:1789762664.630080:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b0211e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.630103:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b0211e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.630120:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b0211e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.630126:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762664.630129:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b0211e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.630136:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b0211e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.630150:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b0211e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762664.630156:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b0211e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762664.630210:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2365:x1876702014833536:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11027us (196147us total) trans 4294967461 rc 0/0 00000100:00100000:2.0:1789762664.630217:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 45 00000100:00100000:2.0:1789762664.630223:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 187922996381 time 187923070833 state 0x2 ref 0 00000100:00100000:0.0:1789762664.632358:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762664.819249:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188123070833 time 188123152059 state 0x0 ref 1 00000100:00100000:0.0:1789762664.819287:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014834176:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762664.819340:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 10485760, mtime: 0x6aad9c68, atime: 0x6aad9c66, ctime: 0x6aad9c68, blocks: 0x5007 00010000:00010000:0.0:1789762664.819347:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02125 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.819360:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=188s v=(0 0) 00010000:00010000:0.0:1789762664.819364:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762664.819369:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.819403:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762664.831886:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:0.0:1789762664.831892:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:0.0:1789762664.831903:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02125 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.831913:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02125 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.831920:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.831926:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762664.831930:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02125 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.831943:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02125 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.831960:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02125 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762664.831965:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a2aedfa/0x416193e012b02125 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762664.832038:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014834176:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12756us (199678us total) trans 4294967462 rc 0/0 00000100:00100000:0.0:1789762664.832047:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 46 00000100:00100000:0.0:1789762664.832053:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188123070833 time 188123152059 state 0x2 ref 0 00000100:00100000:2.0:1789762664.834268:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762665.019337:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188323152059 time 188323240224 state 0x0 ref 1 00000100:00100000:2.0:1789762665.019391:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2365:x1876702014834304:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762665.019467:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 11534336, mtime: 0x6aad9c68, atime: 0x6aad9c66, ctime: 0x6aad9c68, blocks: 0x5807 00010000:00010000:2.0:1789762665.019476:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d45f999f/0x416193e012b0212c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.019497:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=188s v=(0 0) 00010000:00010000:2.0:1789762665.019501:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762665.019506:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d45f999f/0x416193e012b0212c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.019517:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d45f999f/0x416193e012b0212c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762665.030798:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 11534336 -> 12582912 00002000:00010000:0.0:1789762665.030804:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb mtime from disk: 1789762664 -> 1789762665 00002000:00010000:0.0:1789762665.030806:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb ctime from disk: 1789762664 -> 1789762665 00002000:00010000:0.0:1789762665.030808:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 22535 -> 24583 00010000:00010000:0.0:1789762665.030818:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d45f999f/0x416193e012b0212c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.030830:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d45f999f/0x416193e012b0212c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.030841:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d45f999f/0x416193e012b0212c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.030856:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762665.030863:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d45f999f/0x416193e012b0212c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.030872:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d45f999f/0x416193e012b0212c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.030887:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d45f999f/0x416193e012b0212c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.030893:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d45f999f/0x416193e012b0212c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762665.030957:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2365:x1876702014834304:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11601us (196686us total) trans 4294967463 rc 0/0 00000100:00100000:0.0:1789762665.030970:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 47 00000100:00100000:0.0:1789762665.030977:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188323152059 time 188323240224 state 0x2 ref 0 00000100:00100000:0.0:1789762665.033788:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014834432 00000100:00100000:0.0:1789762665.033795:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014834432 00000100:00100000:0.0:1789762665.033830:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188323152059 time 188323240224 state 0x0 ref 1 00000100:00100000:0.0:1789762665.033841:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188523240224 time 188323240224 state 0x1 ref 1 00000100:00100000:0.0:1789762665.079260:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebf51f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022131456:0@lo:400:kworker.0 00000100:00100000:0.0:1789762665.079337:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebf51fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022131200:0@lo:400:kworker.0 00000100:00100000:0.0:1789762665.079404:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762665.079449:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc426a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022131712:0@lo:400:kworker.0 00000100:00100000:0.0:1789762665.079493:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762665.079584:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022131328 00000100:00100000:0.0:1789762665.079589:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022131328 00000100:00100000:0.0:1789762665.079629:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022131712 00000100:00100000:0.0:1789762665.079633:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022131712 00000100:00100000:0.0:1789762665.079663:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 105 00000100:00100000:2.0:1789762665.079672:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 106 00000100:00100000:0.0:1789762665.079676:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e19500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022131328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762665.079686:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022131712:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762665.079753:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e19500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022131328:12345-0@lo:400:kworker.0 Request processed in 79us (432us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762665.079755:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022131712:12345-0@lo:400:kworker.0 Request processed in 71us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762665.079771:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 105 00000100:00100000:2.0:1789762665.079779:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 106 00000100:00100000:0.0:1789762665.079808:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 107 00000100:00100000:0.0:1789762665.079820:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e1b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022131456:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762665.079837:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 56 00000100:00100000:2.0:1789762665.079848:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e1a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022131840:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762665.079867:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e1b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022131456:12345-0@lo:400:kworker.0 Request processed in 51us (154us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762665.079874:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 107 00000100:00100000:2.0:1789762665.079895:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e1a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022131840:12345-0@lo:400:kworker.0 Request processed in 49us (462us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762665.079902:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 56 00000100:00100000:0.0:1789762665.079917:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebf51f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022131456:0@lo:400:kworker.0 00000100:00100000:0.0:1789762665.079958:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc2d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022131840:0@lo:400:kworker.0 00000100:00100000:0.0:1789762665.079978:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebf51ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022131328:0@lo:400:kworker.0 00000001:00080000:0.0:1789762665.088900:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967394 is committed 00000001:00080000:0.0:1789762665.088915:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967398 is committed 00000004:00080000:0.0:1789762665.088940:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 8 committed 00000001:00080000:0.0:1789762665.088950:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967402 is committed 00000004:00080000:0.0:1789762665.088973:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 6 committed 00000004:00080000:0.0:1789762665.088990:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 7 committed 00000100:00100000:0.0:1789762665.089073:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022131968:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789762665.089160:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762665.089194:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022131968 00000100:00100000:2.0:1789762665.089202:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022131968 00010000:00010000:0.0:1789762665.089479:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762665.089488:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b02133 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.089504:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e88522a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762665.089511:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b02133 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.089525:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b02133 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.089544:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b02133 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.089550:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c81bb73/0x416193e012b02133 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.089565:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762665.089567:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762665.089570:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762665.090384:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022132096 00000100:00100000:2.0:1789762665.090390:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022132096 00000100:00100000:2.0:1789762665.090430:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 88 00000100:00100000:2.0:1789762665.090439:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022132096:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1789762665.090453:0:6881:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:4 count 1 00002000:00010000:2.0:1789762665.090473:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x4:0x0] initial LVB size: 8388608, mtime: 0x6aad9c64, atime: 0x6aad9c63, ctime: 0x6aad9c64, blocks: 0x4007 00010000:00010000:2.0:1789762665.090480:0:6881:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000d9ac6f5/0x416193e012b02141 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.090490:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=188s v=(0 0) 00010000:00010000:2.0:1789762665.090494:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x4:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762665.090500:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000d9ac6f5/0x416193e012b02141 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.090513:0:6881:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000d9ac6f5/0x416193e012b02141 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.090522:0:6881:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000000d9ac6f5/0x416193e012b02141 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.090530:0:6881:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000000d9ac6f5/0x416193e012b02141 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.090547:0:6881:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762665.090561:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762665.090567:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000000d9ac6f5/0x416193e012b02141 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.090574:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac30400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762665.090583:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000000d9ac6f5/0x416193e012b02141 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.090590:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000000d9ac6f5/0x416193e012b02141 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.090600:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000000d9ac6f5/0x416193e012b02141 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.090605:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000d9ac6f5/0x416193e012b02141 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x4:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.090618:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762665.090619:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762665.090621:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762665.091118:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022132096:12345-0@lo:6:osp-syn-1-0.0 Request processed in 679us (778us total) trans 4294967343 rc 0/0 00000100:00100000:2.0:1789762665.091127:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 88 00000100:00100000:0.0:1789762665.091256:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762665.091287:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762665.091330:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014834560 00000100:00100000:2.0:1789762665.091336:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014834560 00010000:00010000:2.0:1789762665.091363:0:6664:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b404de1b/0x416193e012b01f0a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd193 expref: 12 pid: 6881 timeout: 288 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762665.091394:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762665.091409:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014834816 00000100:00100000:2.0:1789762665.091414:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014834816 00010000:00010000:2.0:1789762665.091429:0:6664:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b01f11 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1a1 expref: 13 pid: 6649 timeout: 288 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762665.091498:0:6664:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 47 00000100:00100000:2.0:1789762665.091518:0:6664:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2364:x1876702014834816:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762665.091533:0:6664:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762665.091544:0:6664:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b01f11 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1a1 expref: 12 pid: 6649 timeout: 288 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762665.091562:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762665.091575:0:6664:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b01f11 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1a1 expref: 11 pid: 6649 timeout: 288 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.091595:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b01f11 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x6:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1a1 expref: 10 pid: 6649 timeout: 288 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.091610:0:6664:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762665.091643:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762665.091644:0:6664:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdff100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014834816:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 128us (249us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762665.091655:0:6664:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 47 00000100:00100000:2.0:1789762665.091679:0:6664:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 48 00000100:00100000:2.0:1789762665.091692:0:6664:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86f01850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014834944:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762665.091711:0:6664:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762665.091719:0:6664:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b01f18 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1a8 expref: 9 pid: 6647 timeout: 288 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.091733:0:6664:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b01f18 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x6:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1a8 expref: 9 pid: 6647 timeout: 288 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.091748:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b01f18 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x6:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1a8 expref: 9 pid: 6647 timeout: 288 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762665.091786:0:6664:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762665.091822:0:6664:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86f01850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2364:x1876702014834944:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 131us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762665.091831:0:6664:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 48 00000004:00080000:0.0:1789762665.092501:0:3272:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebfdfaa00/2, rc 0, transno 4294967465 00000100:00100000:0.0:1789762665.092509:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022131968:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789762665.219412:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188523240224 time 188523312262 state 0x0 ref 1 00000100:00100000:0.0:1789762665.219445:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014834432:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762665.219495:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 12582912, mtime: 0x6aad9c69, atime: 0x6aad9c66, ctime: 0x6aad9c69, blocks: 0x6007 00010000:00010000:0.0:1789762665.219515:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02148 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.219528:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=188s v=(0 0) 00010000:00010000:0.0:1789762665.219531:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762665.219536:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.219548:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762665.232534:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 12582912 -> 13631488 00002000:00010000:0.0:1789762665.232539:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:0.0:1789762665.232554:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02148 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.232569:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02148 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.232580:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02148 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.232590:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762665.232595:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02148 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.232605:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02148 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.232631:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02148 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.232641:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02148 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762665.232717:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014834432:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13283us (198961us total) trans 4294967466 rc 0/0 00000100:00100000:0.0:1789762665.232728:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 48 00000100:00100000:0.0:1789762665.232736:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188523240224 time 188523312262 state 0x2 ref 0 00000100:00100000:2.0:1789762665.234740:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762665.419472:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188723312262 time 188723377154 state 0x0 ref 1 00000100:00100000:0.0:1789762665.419501:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014835200:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762665.419547:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 13631488, mtime: 0x6aad9c69, atime: 0x6aad9c66, ctime: 0x6aad9c69, blocks: 0x6807 00010000:00010000:0.0:1789762665.419554:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0214f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.419568:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=188s v=(0 0) 00010000:00010000:0.0:1789762665.419572:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762665.419579:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0214f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.419594:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0214f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762665.430582:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967467 is committed 00002000:00010000:0.0:1789762665.430595:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:0.0:1789762665.430601:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:0.0:1789762665.430614:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0214f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.430626:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0214f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.430632:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0214f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.430637:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762665.430641:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0214f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.430648:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0214f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.430662:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0214f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.430668:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0214f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762665.430721:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014835200:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11228us (195981us total) trans 4294967467 rc 0/0 00000100:00100000:0.0:1789762665.430735:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 49 00000100:00100000:0.0:1789762665.430741:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188723312262 time 188723377154 state 0x2 ref 0 00000100:00100000:0.0:1789762665.432737:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014835328 00000100:00100000:0.0:1789762665.432745:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014835328 00000100:00100000:0.0:1789762665.432776:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188723312262 time 188723377154 state 0x0 ref 1 00000100:00100000:0.0:1789762665.432786:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188923377154 time 188723377154 state 0x1 ref 1 00000100:00100000:0.0:1789762665.619556:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188923377154 time 188923456366 state 0x0 ref 1 00000100:00100000:0.0:1789762665.619588:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014835328:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762665.619642:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 14680064, mtime: 0x6aad9c69, atime: 0x6aad9c66, ctime: 0x6aad9c69, blocks: 0x7007 00010000:00010000:0.0:1789762665.619649:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02156 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.619662:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=188s v=(0 0) 00010000:00010000:0.0:1789762665.619665:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762665.619670:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.619681:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762665.631688:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967468 is committed 00002000:00010000:0.0:1789762665.631704:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:0.0:1789762665.631709:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:0.0:1789762665.631718:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02156 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.631733:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02156 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.631741:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02156 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.631747:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762665.631750:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02156 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.631757:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02156 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.631770:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02156 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.631775:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02156 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762665.631829:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014835328:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12255us (199131us total) trans 4294967468 rc 0/0 00000100:00100000:0.0:1789762665.631837:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 50 00000100:00100000:0.0:1789762665.631842:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188923377154 time 188923456366 state 0x2 ref 0 00000100:00100000:2.0:1789762665.633961:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014835456 00000100:00100000:2.0:1789762665.633970:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014835456 00000100:00100000:2.0:1789762665.634004:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 188923377154 time 188923456366 state 0x0 ref 1 00000100:00100000:2.0:1789762665.634014:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189123456366 time 188923456366 state 0x1 ref 1 00000100:00100000:0.0:1789762665.819631:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189123456366 time 189123533721 state 0x0 ref 1 00000100:00100000:0.0:1789762665.819672:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014835456:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762665.819723:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 15728640, mtime: 0x6aad9c69, atime: 0x6aad9c66, ctime: 0x6aad9c69, blocks: 0x7807 00010000:00010000:0.0:1789762665.819730:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0215d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.819744:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=189s v=(0 0) 00010000:00010000:0.0:1789762665.819747:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762665.819752:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0215d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.819764:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0215d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762665.832275:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967469 is committed 00002000:00010000:0.0:1789762665.832293:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:0.0:1789762665.832298:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:0.0:1789762665.832307:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0215d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.832316:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0215d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.832323:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0215d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.832328:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762665.832332:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0215d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.832345:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0215d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.832359:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0215d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762665.832378:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0215d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762665.832435:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014835456:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12770us (198560us total) trans 4294967469 rc 0/0 00000100:00100000:0.0:1789762665.832443:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 51 00000100:00100000:0.0:1789762665.832449:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189123456366 time 189123533721 state 0x2 ref 0 00000100:00100000:2.0:1789762665.834468:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762665.834503:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014835584 00000100:00100000:0.0:1789762665.834512:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014835584 00000100:00100000:0.0:1789762665.834544:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189123456366 time 189123533721 state 0x0 ref 1 00000100:00100000:0.0:1789762665.834553:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189323533721 time 189123533721 state 0x1 ref 1 00000100:00100000:0.0:1789762666.019741:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189323533721 time 189323643237 state 0x0 ref 1 00000100:00100000:0.0:1789762666.019769:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014835584:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762666.019819:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 16777216, mtime: 0x6aad9c69, atime: 0x6aad9c66, ctime: 0x6aad9c69, blocks: 0x8007 00010000:00010000:0.0:1789762666.019826:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02164 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.019855:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=189s v=(0 0) 00010000:00010000:0.0:1789762666.019858:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762666.019864:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.019875:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762666.035582:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967470 is committed 00002000:00010000:0.0:1789762666.035605:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 16777216 -> 17825792 00002000:00010000:0.0:1789762666.035614:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb mtime from disk: 1789762665 -> 1789762666 00002000:00010000:0.0:1789762666.035618:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb ctime from disk: 1789762665 -> 1789762666 00002000:00010000:0.0:1789762666.035621:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:0.0:1789762666.035636:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02164 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.035651:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02164 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.035663:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02164 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.035689:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762666.035694:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02164 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.035704:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02164 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.035738:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02164 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.035748:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02164 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762666.035844:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014835584:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16086us (201375us total) trans 4294967470 rc 0/0 00000100:00100000:0.0:1789762666.035851:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 52 00000100:00100000:0.0:1789762666.035857:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189323533721 time 189323643237 state 0x2 ref 0 00000100:00100000:2.0:1789762666.038288:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762666.219829:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189523643237 time 189523727262 state 0x0 ref 1 00000100:00100000:0.0:1789762666.219865:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014835712:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762666.219935:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 17825792, mtime: 0x6aad9c6a, atime: 0x6aad9c66, ctime: 0x6aad9c6a, blocks: 0x8807 00010000:00010000:0.0:1789762666.219944:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0216b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.219962:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=189s v=(0 0) 00010000:00010000:0.0:1789762666.219966:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762666.219972:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0216b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.219985:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0216b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762666.233219:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:0.0:1789762666.233229:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 34823 -> 36871 00000001:00080000:2.0:1789762666.233233:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967471 is committed 00010000:00010000:0.0:1789762666.233243:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0216b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.233255:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0216b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.233265:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0216b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.233274:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762666.233280:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0216b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.233293:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0216b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.233315:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0216b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.233325:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0216b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762666.233416:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014835712:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13566us (195129us total) trans 4294967471 rc 0/0 00000100:00100000:0.0:1789762666.233434:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 53 00000100:00100000:0.0:1789762666.233442:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189523643237 time 189523727262 state 0x2 ref 0 00000100:00100000:2.0:1789762666.235464:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762666.235506:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014835840 00000100:00100000:0.0:1789762666.235516:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014835840 00000100:00100000:0.0:1789762666.235561:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189523643237 time 189523727262 state 0x0 ref 1 00000100:00100000:0.0:1789762666.235572:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189723727262 time 189523727262 state 0x1 ref 1 00000100:00100000:0.0:1789762666.419923:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189723727262 time 189723821356 state 0x0 ref 1 00000100:00100000:0.0:1789762666.419957:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014835840:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762666.420029:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 18874368, mtime: 0x6aad9c6a, atime: 0x6aad9c66, ctime: 0x6aad9c6a, blocks: 0x9007 00010000:00010000:0.0:1789762666.420038:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02172 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.420055:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=189s v=(0 0) 00010000:00010000:0.0:1789762666.420060:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762666.420065:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.420078:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.423420:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762666.435058:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967472 is committed 00002000:00010000:0.0:1789762666.435079:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:0.0:1789762666.435112:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 36871 -> 38921 00010000:00010000:0.0:1789762666.435135:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02172 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.435151:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02172 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.435162:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02172 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.435171:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762666.435177:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02172 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.435189:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02172 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.435209:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02172 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.435219:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02172 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762666.435309:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014835840:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15363us (199840us total) trans 4294967472 rc 0/0 00000100:00100000:0.0:1789762666.435322:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 54 00000100:00100000:0.0:1789762666.435332:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189723727262 time 189723821356 state 0x2 ref 0 00000100:00100000:2.0:1789762666.437418:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762666.620002:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189923821356 time 189923904617 state 0x0 ref 1 00000100:00100000:0.0:1789762666.620040:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014835968:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762666.620113:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 19922944, mtime: 0x6aad9c6a, atime: 0x6aad9c66, ctime: 0x6aad9c6a, blocks: 0x9809 00010000:00010000:0.0:1789762666.620120:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02179 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.620134:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=189s v=(0 0) 00010000:00010000:0.0:1789762666.620138:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762666.620142:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.620153:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762666.632880:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967473 is committed 00002000:00010000:0.0:1789762666.632910:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 19922944 -> 20971520 00002000:00010000:0.0:1789762666.632917:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 38921 -> 40969 00010000:00010000:0.0:1789762666.632930:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02179 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.632943:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02179 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.632954:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02179 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.632963:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762666.632969:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02179 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.632988:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02179 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.633008:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02179 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.633018:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02179 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762666.633122:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014835968:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13053us (195671us total) trans 4294967473 rc 0/0 00000100:00100000:0.0:1789762666.633135:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 55 00000100:00100000:0.0:1789762666.633144:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189923821356 time 189923904617 state 0x2 ref 0 00000100:00100000:0.0:1789762666.635148:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014836096 00000100:00100000:0.0:1789762666.635157:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014836096 00000100:00100000:0.0:1789762666.635194:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 189923821356 time 189923904617 state 0x0 ref 1 00000100:00100000:0.0:1789762666.635207:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190123904617 time 189923904617 state 0x1 ref 1 00000100:00100000:0.0:1789762666.820152:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190123904617 time 190124051886 state 0x0 ref 1 00000100:00100000:0.0:1789762666.820187:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014836096:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762666.820248:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 20971520, mtime: 0x6aad9c6a, atime: 0x6aad9c66, ctime: 0x6aad9c6a, blocks: 0xa009 00010000:00010000:0.0:1789762666.820257:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02180 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.820297:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=190s v=(0 0) 00010000:00010000:0.0:1789762666.820301:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762666.820308:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.820324:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762666.832667:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967474 is committed 00002000:00010000:0.0:1789762666.832696:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:0.0:1789762666.832702:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 40969 -> 43017 00010000:00010000:0.0:1789762666.832715:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02180 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.832728:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02180 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.832739:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02180 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.832748:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762666.832753:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02180 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.832763:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02180 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.832796:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02180 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762666.832806:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02180 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762666.832879:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014836096:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12706us (197788us total) trans 4294967474 rc 0/0 00000100:00100000:0.0:1789762666.832890:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 56 00000100:00100000:0.0:1789762666.832898:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190123904617 time 190124051886 state 0x2 ref 0 00000100:00100000:2.0:1789762666.834926:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014836224 00000100:00100000:2.0:1789762666.834934:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014836224 00000100:00100000:2.0:1789762666.834969:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190123904617 time 190124051886 state 0x0 ref 1 00000100:00100000:2.0:1789762666.834978:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190324051886 time 190124051886 state 0x1 ref 1 00000100:00100000:0.0:1789762667.020236:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190324051886 time 190324138969 state 0x0 ref 1 00000100:00100000:0.0:1789762667.020279:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014836224:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762667.020333:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 22020096, mtime: 0x6aad9c6a, atime: 0x6aad9c66, ctime: 0x6aad9c6a, blocks: 0xa809 00010000:00010000:0.0:1789762667.020342:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02187 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.020367:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=190s v=(0 0) 00010000:00010000:0.0:1789762667.020372:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762667.020377:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.020391:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762667.033239:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967475 is committed 00002000:00010000:0.0:1789762667.033242:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 22020096 -> 23068672 00002000:00010000:0.0:1789762667.033247:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb mtime from disk: 1789762666 -> 1789762667 00002000:00010000:0.0:1789762667.033250:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb ctime from disk: 1789762666 -> 1789762667 00002000:00010000:0.0:1789762667.033252:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 43017 -> 45065 00010000:00010000:0.0:1789762667.033261:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02187 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.033270:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02187 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.033277:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02187 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.033282:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762667.033286:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02187 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.033292:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02187 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.033306:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02187 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.033312:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02187 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762667.033390:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014836224:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13137us (198511us total) trans 4294967475 rc 0/0 00000100:00100000:0.0:1789762667.033401:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 57 00000100:00100000:0.0:1789762667.033408:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190324051886 time 190324138969 state 0x2 ref 0 00000100:00100000:0.0:1789762667.035330:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014836352 00000100:00100000:0.0:1789762667.035339:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014836352 00000100:00100000:0.0:1789762667.035385:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190324051886 time 190324138969 state 0x0 ref 1 00000100:00100000:0.0:1789762667.035397:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190524138969 time 190324138969 state 0x1 ref 1 00000100:00100000:0.0:1789762667.220315:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190524138969 time 190524215844 state 0x0 ref 1 00000100:00100000:0.0:1789762667.220347:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014836352:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762667.220418:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 23068672, mtime: 0x6aad9c6b, atime: 0x6aad9c66, ctime: 0x6aad9c6b, blocks: 0xb009 00010000:00010000:0.0:1789762667.220428:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0218e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.220446:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=190s v=(0 0) 00010000:00010000:0.0:1789762667.220451:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762667.220457:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0218e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.220473:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0218e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762667.232345:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967476 is committed 00002000:00010000:0.0:1789762667.232376:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:0.0:1789762667.232383:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 45065 -> 47113 00010000:00010000:0.0:1789762667.232395:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0218e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.232408:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0218e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.232419:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0218e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.232427:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762667.232432:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0218e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.232441:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0218e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.232457:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0218e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.232462:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0218e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762667.232517:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014836352:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12182us (197218us total) trans 4294967476 rc 0/0 00000100:00100000:0.0:1789762667.232530:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 58 00000100:00100000:0.0:1789762667.232536:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190524138969 time 190524215844 state 0x2 ref 0 00000100:00100000:2.0:1789762667.234602:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014836480 00000100:00100000:2.0:1789762667.234610:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014836480 00000100:00100000:2.0:1789762667.234643:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190524138969 time 190524215844 state 0x0 ref 1 00000100:00100000:2.0:1789762667.234653:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190724215844 time 190524215844 state 0x1 ref 1 00000100:00100000:0.0:1789762667.420441:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190724215844 time 190724341999 state 0x0 ref 1 00000100:00100000:0.0:1789762667.420469:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014836480:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762667.420525:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 24117248, mtime: 0x6aad9c6b, atime: 0x6aad9c66, ctime: 0x6aad9c6b, blocks: 0xb809 00010000:00010000:0.0:1789762667.420536:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02195 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.420553:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=190s v=(0 0) 00010000:00010000:0.0:1789762667.420558:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762667.420564:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.420590:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762667.432642:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967477 is committed 00002000:00010000:0.0:1789762667.432670:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 24117248 -> 25165824 00002000:00010000:0.0:1789762667.432677:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 47113 -> 49161 00010000:00010000:0.0:1789762667.432689:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02195 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.432714:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02195 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.432738:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02195 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.432746:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762667.432751:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02195 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.432761:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02195 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.432780:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02195 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.432789:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b02195 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762667.432869:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014836480:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12408us (198314us total) trans 4294967477 rc 0/0 00000100:00100000:0.0:1789762667.432880:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 59 00000100:00100000:0.0:1789762667.432888:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190724215844 time 190724341999 state 0x2 ref 0 00000100:00100000:2.0:1789762667.434923:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762667.434969:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014836608 00000100:00100000:0.0:1789762667.434979:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014836608 00000100:00100000:0.0:1789762667.435018:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190724215844 time 190724341999 state 0x0 ref 1 00000100:00100000:0.0:1789762667.435030:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190924341999 time 190724341999 state 0x1 ref 1 00010000:00010000:0.0:1789762667.447255:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762667.620546:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190924341999 time 190924446334 state 0x0 ref 1 00000100:00100000:0.0:1789762667.620585:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014836608:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762667.620647:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 25165824, mtime: 0x6aad9c6b, atime: 0x6aad9c66, ctime: 0x6aad9c6b, blocks: 0xc009 00010000:00010000:0.0:1789762667.620658:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0219c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.620676:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=190s v=(0 0) 00010000:00010000:0.0:1789762667.620681:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762667.620688:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0219c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.620703:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0219c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762667.636883:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:0.0:1789762667.636889:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 49161 -> 51209 00010000:00010000:0.0:1789762667.636922:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0219c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.636936:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0219c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.636945:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0219c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.636967:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762667.636972:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0219c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.636982:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0219c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.637018:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0219c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.637024:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b0219c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762667.637165:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014836608:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16593us (202236us total) trans 4294967478 rc 0/0 00000100:00100000:0.0:1789762667.637178:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 60 00000100:00100000:0.0:1789762667.637184:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190924341999 time 190924446334 state 0x2 ref 0 00000100:00100000:2.0:1789762667.639852:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014836736 00000100:00100000:2.0:1789762667.639864:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014836736 00000100:00100000:2.0:1789762667.639908:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 190924341999 time 190924446334 state 0x0 ref 1 00000100:00100000:2.0:1789762667.639920:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191124446334 time 190924446334 state 0x1 ref 1 00000100:00100000:0.0:1789762667.820630:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191124446334 time 191124533161 state 0x0 ref 1 00000100:00100000:0.0:1789762667.820659:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014836736:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762667.820708:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 26214400, mtime: 0x6aad9c6b, atime: 0x6aad9c66, ctime: 0x6aad9c6b, blocks: 0xc809 00010000:00010000:0.0:1789762667.820716:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.820755:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=191s v=(0 0) 00010000:00010000:0.0:1789762667.820758:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762667.820763:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.820776:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762667.832535:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967479 is committed 00002000:00010000:0.0:1789762667.832559:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 26214400 -> 27262976 00002000:00010000:0.0:1789762667.832565:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 51209 -> 53257 00010000:00010000:0.0:1789762667.832578:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.832593:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.832603:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.832610:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762667.832615:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.832625:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.832662:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762667.832671:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762667.832750:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014836736:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12101us (192964us total) trans 4294967479 rc 0/0 00000100:00100000:0.0:1789762667.832763:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 61 00000100:00100000:0.0:1789762667.832770:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191124446334 time 191124533161 state 0x2 ref 0 00000100:00100000:2.0:1789762667.835072:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014836864 00000100:00100000:2.0:1789762667.835080:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014836864 00000100:00100000:2.0:1789762667.835130:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191124446334 time 191124533161 state 0x0 ref 1 00000100:00100000:2.0:1789762667.835149:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191324533161 time 191124533161 state 0x1 ref 1 00000100:00100000:0.0:1789762668.020721:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191324533161 time 191324624068 state 0x0 ref 1 00000100:00100000:0.0:1789762668.020748:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014836864:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762668.020796:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 27262976, mtime: 0x6aad9c6b, atime: 0x6aad9c66, ctime: 0x6aad9c6b, blocks: 0xd009 00010000:00010000:0.0:1789762668.020804:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.020817:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=191s v=(0 0) 00010000:00010000:0.0:1789762668.020820:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762668.020825:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.020836:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762668.032807:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:0.0:1789762668.032813:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb mtime from disk: 1789762667 -> 1789762668 00002000:00010000:0.0:1789762668.032815:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb ctime from disk: 1789762667 -> 1789762668 00002000:00010000:0.0:1789762668.032817:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 53257 -> 55305 00000001:00080000:2.0:1789762668.032822:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967480 is committed 00010000:00010000:0.0:1789762668.032827:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.032836:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.032844:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.032849:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762668.032853:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.032860:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.032874:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.032879:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762668.032943:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014836864:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12205us (197922us total) trans 4294967480 rc 0/0 00000100:00100000:0.0:1789762668.032950:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 62 00000100:00100000:0.0:1789762668.032956:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191324533161 time 191324624068 state 0x2 ref 0 00000100:00100000:2.0:1789762668.035072:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762668.220863:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191524624068 time 191524764759 state 0x0 ref 1 00000100:00100000:0.0:1789762668.220890:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014836992:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762668.220948:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 28311552, mtime: 0x6aad9c6c, atime: 0x6aad9c66, ctime: 0x6aad9c6c, blocks: 0xd809 00010000:00010000:0.0:1789762668.220958:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.220976:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=191s v=(0 0) 00010000:00010000:0.0:1789762668.220980:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762668.220985:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.220997:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762668.233737:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967481 is committed 00002000:00010000:0.0:1789762668.233759:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:0.0:1789762668.233764:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 55305 -> 57353 00010000:00010000:0.0:1789762668.233776:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.233802:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.233812:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.233820:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762668.233825:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.233835:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.233853:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.233862:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762668.233923:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014836992:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13043us (198849us total) trans 4294967481 rc 0/0 00000100:00100000:0.0:1789762668.233933:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 63 00000100:00100000:0.0:1789762668.233940:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191524624068 time 191524764759 state 0x2 ref 0 00000100:00100000:2.0:1789762668.237358:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762668.420965:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191724764759 time 191724866424 state 0x0 ref 1 00000100:00100000:0.0:1789762668.421005:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014837120:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762668.421058:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 29360128, mtime: 0x6aad9c6c, atime: 0x6aad9c66, ctime: 0x6aad9c6c, blocks: 0xe009 00010000:00010000:0.0:1789762668.421068:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.421121:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=191s v=(0 0) 00010000:00010000:0.0:1789762668.421126:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762668.421133:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.421149:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762668.432284:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:0.0:1789762668.432291:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 57353 -> 59401 00010000:00010000:0.0:1789762668.432302:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.432315:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.432325:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.432333:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762668.432339:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.432372:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.432391:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.432400:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762668.432456:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014837120:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11459us (195109us total) trans 4294967482 rc 0/0 00000100:00100000:0.0:1789762668.432466:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 64 00000100:00100000:0.0:1789762668.432473:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191724764759 time 191724866424 state 0x2 ref 0 00000100:00100000:2.0:1789762668.434788:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014837248 00000100:00100000:2.0:1789762668.434798:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014837248 00000100:00100000:2.0:1789762668.434837:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191724764759 time 191724866424 state 0x0 ref 1 00000100:00100000:2.0:1789762668.434849:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191924866424 time 191724866424 state 0x1 ref 1 00000100:00100000:0.0:1789762668.621046:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191924866424 time 191924946556 state 0x0 ref 1 00000100:00100000:0.0:1789762668.621079:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014837248:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762668.621149:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 30408704, mtime: 0x6aad9c6c, atime: 0x6aad9c66, ctime: 0x6aad9c6c, blocks: 0xe809 00010000:00010000:0.0:1789762668.621158:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.621172:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=191s v=(0 0) 00010000:00010000:0.0:1789762668.621175:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762668.621180:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.621209:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762668.632466:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 30408704 -> 31457280 00002000:00010000:0.0:1789762668.632471:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 59401 -> 61449 00010000:00010000:0.0:1789762668.632481:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.632490:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.632496:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.632501:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762668.632505:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.632512:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.632526:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.632534:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762668.632614:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014837248:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11548us (197866us total) trans 4294967483 rc 0/0 00000100:00100000:0.0:1789762668.632624:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 65 00000100:00100000:0.0:1789762668.632632:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 191924866424 time 191924946556 state 0x2 ref 0 00000100:00100000:2.0:1789762668.635879:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762668.821131:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 192124946556 time 192125034333 state 0x0 ref 1 00000100:00100000:0.0:1789762668.821157:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014837376:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762668.821205:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 31457280, mtime: 0x6aad9c6c, atime: 0x6aad9c66, ctime: 0x6aad9c6c, blocks: 0xf009 00010000:00010000:0.0:1789762668.821213:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.821226:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=192s v=(0 0) 00010000:00010000:0.0:1789762668.821229:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762668.821234:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.821248:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762668.832493:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967484 is committed 00002000:00010000:0.0:1789762668.832513:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:0.0:1789762668.832518:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 61449 -> 63497 00010000:00010000:0.0:1789762668.832527:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.832547:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.832554:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.832559:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762668.832562:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.832570:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.832583:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762668.832588:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762668.832640:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014837376:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11494us (196766us total) trans 4294967484 rc 0/0 00000100:00100000:0.0:1789762668.832648:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 66 00000100:00100000:0.0:1789762668.832653:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 192124946556 time 192125034333 state 0x2 ref 0 00000100:00100000:2.0:1789762668.835011:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762669.021213:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 192325034333 time 192325115316 state 0x0 ref 1 00000100:00100000:0.0:1789762669.021266:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014837504:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762669.021317:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 32505856, mtime: 0x6aad9c6c, atime: 0x6aad9c66, ctime: 0x6aad9c6c, blocks: 0xf809 00010000:00010000:0.0:1789762669.021325:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.021338:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=192s v=(0 0) 00010000:00010000:0.0:1789762669.021355:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762669.021361:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.021371:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b021cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762669.036619:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967485 is committed 00000100:00100000:2.0:1789762669.039267:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762669.221427:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 192525115316 time 192525323838 state 0x0 ref 1 00000100:00100000:0.0:1789762669.221469:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014837632:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762669.221539:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 33554432, mtime: 0x6aad9c6d, atime: 0x6aad9c66, ctime: 0x6aad9c6d, blocks: 0x10009 00010000:00010000:0.0:1789762669.221550:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b021d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.221567:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=192s v=(0 0) 00010000:00010000:0.0:1789762669.221572:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762669.221578:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b021d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.221637:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b021d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762669.239739:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:0.0:1789762669.239749:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 65545 -> 67593 00010000:00010000:0.0:1789762669.239768:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b021d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.239785:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b021d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.239798:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b021d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.239824:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812ca00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762669.239829:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b021d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.239840:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b021d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.239860:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b021d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.239868:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dfac3806/0x416193e012b021d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762669.239990:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014837632:12345-192.168.203.48@tcp:4:dd.500 Request processed in 18532us (200713us total) trans 4294967486 rc 0/0 00000100:00100000:0.0:1789762669.240007:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 68 00000100:00100000:0.0:1789762669.240018:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 192525115316 time 192525323838 state 0x2 ref 0 00000100:00100000:2.0:1789762669.243160:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00002000:00010000:0.0:1789762669.438081:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:0.0:1789762669.438123:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 67593 -> 69641 00010000:00010000:0.0:1789762669.438141:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006a75efda/0x416193e012b021db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.438163:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006a75efda/0x416193e012b021db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.438179:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006a75efda/0x416193e012b021db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.438192:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e88523600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762669.438197:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006a75efda/0x416193e012b021db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.438208:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006a75efda/0x416193e012b021db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.438229:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006a75efda/0x416193e012b021db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.438256:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006a75efda/0x416193e012b021db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762669.438369:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014837760:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16790us (195208us total) trans 4294967487 rc 0/0 00000100:00100000:0.0:1789762669.438400:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 69 00000100:00100000:0.0:1789762669.438410:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 192725323838 time 192725434949 state 0x2 ref 0 00000100:00100000:2.0:1789762669.440716:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762669.621649:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 192925434949 time 192925549288 state 0x0 ref 1 00000100:00100000:0.0:1789762669.621685:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014837888:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762669.621750:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 35651584, mtime: 0x6aad9c6d, atime: 0x6aad9c66, ctime: 0x6aad9c6d, blocks: 0x11009 00010000:00010000:0.0:1789762669.621760:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.621804:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=192s v=(0 0) 00010000:00010000:0.0:1789762669.621813:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762669.621823:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.621846:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762669.636413:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 35651584 -> 36700160 00002000:00010000:0.0:1789762669.636419:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 69641 -> 71689 00010000:00010000:0.0:1789762669.636438:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.636450:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.636458:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.636464:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762669.636468:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.636477:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.636493:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.636499:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762669.636563:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014837888:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14892us (195849us total) trans 4294967488 rc 0/0 00000100:00100000:0.0:1789762669.636573:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 70 00000100:00100000:0.0:1789762669.636579:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 192925434949 time 192925549288 state 0x2 ref 0 00000100:00100000:2.0:1789762669.638747:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762669.666178:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762669.666220:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762669.666244:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762669.666274:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762669.666292:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014838528 00000100:00100000:2.0:1789762669.666299:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014838528 00000100:00100000:0.0:1789762669.666304:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014838400 00000100:00100000:0.0:1789762669.666312:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014838400 00000100:00100000:2.0:1789762669.666360:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 109 00000100:00100000:2.0:1789762669.666376:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014838528:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762669.666382:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 57 00000100:00100000:0.0:1789762669.666400:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014838400:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762669.666419:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014838528:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 47us (173us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762669.666440:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 109 00000100:00100000:0.0:1789762669.666461:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014838400:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 65us (184us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762669.666468:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 57 00000100:00100000:0.0:1789762669.687203:0:3272:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789762669.687223:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfed1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022132992:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762669.687279:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762669.687358:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022132992 00000100:00100000:0.0:1789762669.687365:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022132992 00000100:00080000:2.0:1789762669.687403:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762669.687432:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 49 00000100:00100000:0.0:1789762669.687446:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022132992:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762669.687465:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 131137536 pending 0 free 3828350976 avail 3821010944 00000100:00100000:0.0:1789762669.687506:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022132992:12345-0@lo:13:osp-pre-0-0.0 Request processed in 63us (226us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762669.687514:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 49 00000100:00100000:0.0:1789762669.687558:0:3272:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967465 gen 2 00000004:00080000:0.0:1789762669.687562:0:3272:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebfdfa680, transno 4294967464 00000004:00080000:0.0:1789762669.687574:0:3272:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebfdfaa00, transno 4294967465 00000100:00100000:0.0:1789762669.687586:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfed1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022132992:0@lo:13:osp-pre-0-0.0 00000001:00080000:2.0:1789762669.696397:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967343 is committed 00000100:00100000:0.0:1789762669.821732:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193125549288 time 193125635581 state 0x0 ref 1 00000100:00100000:0.0:1789762669.821770:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014838016:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762669.821829:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 36700160, mtime: 0x6aad9c6d, atime: 0x6aad9c66, ctime: 0x6aad9c6d, blocks: 0x11809 00010000:00010000:0.0:1789762669.821838:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.821854:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=193s v=(0 0) 00010000:00010000:0.0:1789762669.821858:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762669.821866:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.821883:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762669.833435:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 36700160 -> 37748736 00002000:00010000:0.0:1789762669.833440:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 71689 -> 73737 00010000:00010000:0.0:1789762669.833450:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.833459:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.833466:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.833471:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762669.833474:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.833489:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.833504:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762669.833509:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762669.833578:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014838016:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11815us (194830us total) trans 4294967489 rc 0/0 00000100:00100000:0.0:1789762669.833586:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 71 00000100:00100000:0.0:1789762669.833592:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193125549288 time 193125635581 state 0x2 ref 0 00000100:00100000:2.0:1789762669.835922:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014838656 00000100:00100000:2.0:1789762669.835931:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014838656 00000100:00100000:2.0:1789762669.835966:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193125549288 time 193125635581 state 0x0 ref 1 00000100:00100000:2.0:1789762669.835975:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193325635581 time 193125635581 state 0x1 ref 1 00000100:00100000:0.0:1789762670.021827:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193325635581 time 193325726221 state 0x0 ref 1 00000100:00100000:0.0:1789762670.021863:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014838656:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762670.021930:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 37748736, mtime: 0x6aad9c6d, atime: 0x6aad9c66, ctime: 0x6aad9c6d, blocks: 0x12009 00010000:00010000:0.0:1789762670.021939:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.021953:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=193s v=(0 0) 00010000:00010000:0.0:1789762670.021957:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762670.021961:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.021990:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762670.033113:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967490 is committed 00002000:00010000:0.0:1789762670.033118:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:0.0:1789762670.033123:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb mtime from disk: 1789762669 -> 1789762670 00002000:00010000:0.0:1789762670.033126:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb ctime from disk: 1789762669 -> 1789762670 00002000:00010000:0.0:1789762670.033128:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 73737 -> 75785 00010000:00010000:0.0:1789762670.033137:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.033147:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.033153:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.033159:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762670.033163:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.033169:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.033183:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.033197:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b021f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762670.033252:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014838656:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11405us (197386us total) trans 4294967490 rc 0/0 00000100:00100000:0.0:1789762670.033260:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 72 00000100:00100000:0.0:1789762670.033265:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193325635581 time 193325726221 state 0x2 ref 0 00000100:00100000:2.0:1789762670.035630:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762670.199238:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdf9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022133248:0@lo:400:kworker.0 00000100:00100000:2.0:1789762670.199306:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762670.199396:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022133632:0@lo:400:kworker.0 00000100:00100000:2.0:1789762670.199450:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022133248 00000100:00100000:2.0:1789762670.199457:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022133248 00000100:00100000:2.0:1789762670.199511:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 249 00000100:00100000:2.0:1789762670.199523:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022133248:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762670.199559:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022133760 00000100:00100000:0.0:1789762670.199566:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022133760 00000100:00100000:2.0:1789762670.199570:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022133248:12345-0@lo:400:kworker.0 Request processed in 49us (263us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762670.199577:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 249 00000100:00100000:0.0:1789762670.199617:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 111 00000100:00100000:0.0:1789762670.199629:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022133760:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762670.199638:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762670.199683:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022133504 00000100:00100000:2.0:1789762670.199690:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022133504 00000100:00100000:0.0:1789762670.199697:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022133760:12345-0@lo:400:kworker.0 Request processed in 69us (223us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762670.199710:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 111 00000100:00100000:0.0:1789762670.199735:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022133888 00000100:00100000:0.0:1789762670.199741:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022133888 00000100:00080000:2.0:1789762670.199761:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762670.199767:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762670.199769:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022133632 00000100:00100000:0.0:1789762670.199773:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022133632 00000100:00100000:0.0:1789762670.199797:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 58 00000100:00100000:0.0:1789762670.199807:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022133888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762670.199861:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022133888:12345-0@lo:400:kworker.0 Request processed in 55us (432us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762670.199869:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 58 00000100:00100000:2.0:1789762670.206296:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdf9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022134016:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789762670.206323:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762670.206353:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022134016 00000100:00100000:0.0:1789762670.206359:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022134016 00000100:00100000:0.0:1789762670.206392:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 89 00000100:00100000:0.0:1789762670.206400:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022134016:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1789762670.206415:0:6647:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:5 count 1 00002000:00010000:0.0:1789762670.206444:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x5:0x0] initial LVB size: 8388608, mtime: 0x6aad9c64, atime: 0x6aad9c63, ctime: 0x6aad9c64, blocks: 0x4007 00010000:00010000:0.0:1789762670.206453:0:6647:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ccbca09b/0x416193e012b021f7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.206470:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=193s v=(0 0) 00010000:00010000:0.0:1789762670.206475:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x5:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762670.206480:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ccbca09b/0x416193e012b021f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.206493:0:6647:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ccbca09b/0x416193e012b021f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.206500:0:6647:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ccbca09b/0x416193e012b021f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.206506:0:6647:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ccbca09b/0x416193e012b021f7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x5:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.206523:0:6647:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1789762670.206902:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022134016:12345-0@lo:6:osp-syn-1-0.0 Request processed in 503us (578us total) trans 4294967344 rc 0/0 00000100:00100000:0.0:1789762670.206908:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 89 00000004:00080000:2.0:1789762670.206929:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebfdf9f80/2, rc 0, transno 4294967344 00000100:00100000:2.0:1789762670.206935:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdf9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022134016:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789762670.221921:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193525726221 time 193525828241 state 0x0 ref 1 00000100:00100000:0.0:1789762670.221944:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014838784:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762670.221973:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 38797312, mtime: 0x6aad9c6e, atime: 0x6aad9c66, ctime: 0x6aad9c6e, blocks: 0x12809 00010000:00010000:0.0:1789762670.221978:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b021fe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.221986:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=193s v=(0 0) 00010000:00010000:0.0:1789762670.221989:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762670.221992:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b021fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.222000:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b021fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762670.232348:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967491 is committed 00002000:00010000:0.0:1789762670.232354:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:0.0:1789762670.232360:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 75785 -> 77833 00010000:00010000:0.0:1789762670.232369:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b021fe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.232378:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b021fe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.232384:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b021fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.232389:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762670.232392:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b021fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.232404:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b021fe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.232415:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b021fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.232421:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b021fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762670.232475:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014838784:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10533us (196845us total) trans 4294967491 rc 0/0 00000100:00100000:0.0:1789762670.232484:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 73 00000100:00100000:0.0:1789762670.232489:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193525726221 time 193525828241 state 0x2 ref 0 00000100:00100000:2.0:1789762670.234604:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014838912 00000100:00100000:2.0:1789762670.234611:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014838912 00000100:00100000:2.0:1789762670.234639:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193525726221 time 193525828241 state 0x0 ref 1 00000100:00100000:2.0:1789762670.234647:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193725828241 time 193525828241 state 0x1 ref 1 00000100:00100000:0.0:1789762670.422040:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193725828241 time 193725941676 state 0x0 ref 1 00000100:00100000:0.0:1789762670.422066:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014838912:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762670.422140:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 39845888, mtime: 0x6aad9c6e, atime: 0x6aad9c66, ctime: 0x6aad9c6e, blocks: 0x13009 00010000:00010000:0.0:1789762670.422148:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02205 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.422161:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=193s v=(0 0) 00010000:00010000:0.0:1789762670.422165:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762670.422169:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.422196:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762670.433966:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967492 is committed 00002000:00010000:0.0:1789762670.433982:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 39845888 -> 40894464 00002000:00010000:0.0:1789762670.433988:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 77833 -> 79881 00010000:00010000:0.0:1789762670.434000:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02205 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.434013:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02205 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.434023:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02205 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.434028:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762670.434032:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02205 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.434039:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02205 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.434054:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02205 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.434062:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02205 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762670.434147:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014838912:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12088us (199586us total) trans 4294967492 rc 0/0 00000100:00100000:0.0:1789762670.434155:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 74 00000100:00100000:0.0:1789762670.434161:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193725828241 time 193725941676 state 0x2 ref 0 00000100:00100000:2.0:1789762670.436751:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762670.519198:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762670.622129:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193925941676 time 193926034154 state 0x0 ref 1 00000100:00100000:0.0:1789762670.622153:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014839040:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762670.622201:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 40894464, mtime: 0x6aad9c6e, atime: 0x6aad9c66, ctime: 0x6aad9c6e, blocks: 0x13809 00010000:00010000:0.0:1789762670.622208:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0220c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.622221:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=193s v=(0 0) 00010000:00010000:0.0:1789762670.622225:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762670.622230:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0220c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.622240:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0220c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762670.632745:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967493 is committed 00002000:00010000:0.0:1789762670.632763:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:0.0:1789762670.632769:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 79881 -> 81929 00010000:00010000:0.0:1789762670.632779:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0220c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.632797:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0220c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.632804:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0220c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.632809:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762670.632812:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0220c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.632820:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0220c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.632833:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0220c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.632838:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0220c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762670.632893:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014839040:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10749us (196141us total) trans 4294967493 rc 0/0 00000100:00100000:0.0:1789762670.632904:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 75 00000100:00100000:0.0:1789762670.632911:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 193925941676 time 193926034154 state 0x2 ref 0 00000100:00100000:2.0:1789762670.635271:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762670.822252:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194126034154 time 194126153913 state 0x0 ref 1 00000100:00100000:0.0:1789762670.822290:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81380e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014839168:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762670.822354:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 41943040, mtime: 0x6aad9c6e, atime: 0x6aad9c66, ctime: 0x6aad9c6e, blocks: 0x14009 00010000:00010000:0.0:1789762670.822362:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02213 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.822375:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=194s v=(0 0) 00010000:00010000:0.0:1789762670.822379:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762670.822384:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.822396:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762670.833300:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 41943040 -> 42991616 00002000:00010000:0.0:1789762670.833305:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 81929 -> 83977 00010000:00010000:0.0:1789762670.833314:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02213 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.833323:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02213 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.833342:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02213 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.833348:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762670.833351:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02213 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.833364:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02213 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.833378:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02213 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762670.833383:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02213 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762670.833435:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81380e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014839168:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11153us (198164us total) trans 4294967494 rc 0/0 00000100:00100000:0.0:1789762670.833443:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 76 00000100:00100000:0.0:1789762670.833448:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194126034154 time 194126153913 state 0x2 ref 0 00000100:00100000:2.0:1789762670.836004:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014839296 00000100:00100000:2.0:1789762670.836012:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014839296 00000100:00100000:2.0:1789762670.836044:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194126034154 time 194126153913 state 0x0 ref 1 00000100:00100000:2.0:1789762670.836053:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194326153913 time 194126153913 state 0x1 ref 1 00000100:00100000:0.0:1789762671.022340:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194326153913 time 194326233368 state 0x0 ref 1 00000100:00100000:0.0:1789762671.022379:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014839296:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:2.0:1789762671.022401:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789762671.022431:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 42991616, mtime: 0x6aad9c6e, atime: 0x6aad9c66, ctime: 0x6aad9c6e, blocks: 0x14809 00010000:00010000:0.0:1789762671.022439:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0221a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.022453:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=194s v=(0 0) 00010000:00010000:0.0:1789762671.022456:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762671.022460:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0221a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.022489:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0221a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762671.033605:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 42991616 -> 44040192 00002000:00010000:0.0:1789762671.033610:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb mtime from disk: 1789762670 -> 1789762671 00002000:00010000:0.0:1789762671.033612:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb ctime from disk: 1789762670 -> 1789762671 00002000:00010000:0.0:1789762671.033614:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 83977 -> 86027 00010000:00010000:0.0:1789762671.033623:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0221a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.033631:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0221a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.033638:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0221a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.033643:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762671.033646:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0221a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.033652:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0221a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.033666:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0221a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.033682:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0221a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762671.033738:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014839296:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11380us (197809us total) trans 4294967495 rc 0/0 00000100:00100000:0.0:1789762671.033746:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 77 00000100:00100000:0.0:1789762671.033751:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194326153913 time 194326233368 state 0x2 ref 0 00000100:00100000:2.0:1789762671.036018:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014839424 00000100:00100000:2.0:1789762671.036026:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014839424 00000100:00100000:2.0:1789762671.036056:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194326153913 time 194326233368 state 0x0 ref 1 00000100:00100000:2.0:1789762671.036065:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194526233368 time 194326233368 state 0x1 ref 1 00000100:00100000:0.0:1789762671.222462:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194526233368 time 194526364439 state 0x0 ref 1 00000100:00100000:0.0:1789762671.222490:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014839424:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762671.222544:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 44040192, mtime: 0x6aad9c6f, atime: 0x6aad9c66, ctime: 0x6aad9c6f, blocks: 0x1500b 00010000:00010000:0.0:1789762671.222551:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02221 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.222564:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=194s v=(0 0) 00010000:00010000:0.0:1789762671.222568:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762671.222572:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.222603:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762671.238074:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 44040192 -> 45088768 00002000:00010000:0.0:1789762671.238080:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 86027 -> 88075 00010000:00010000:0.0:1789762671.238114:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02221 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.238123:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02221 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.238130:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02221 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.238135:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762671.238138:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02221 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.238145:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02221 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.238160:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02221 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.238165:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02221 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762671.238223:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014839424:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15745us (202247us total) trans 4294967496 rc 0/0 00000100:00100000:0.0:1789762671.238231:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 78 00000100:00100000:0.0:1789762671.238237:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194526233368 time 194526364439 state 0x2 ref 0 00000100:00100000:2.0:1789762671.241132:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762671.422538:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194726364439 time 194726440284 state 0x0 ref 1 00000100:00100000:0.0:1789762671.422575:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81380a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014839552:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762671.422637:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 45088768, mtime: 0x6aad9c6f, atime: 0x6aad9c66, ctime: 0x6aad9c6f, blocks: 0x1580b 00010000:00010000:0.0:1789762671.422645:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02228 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.422675:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=194s v=(0 0) 00010000:00010000:0.0:1789762671.422679:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762671.422685:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.422699:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762671.436648:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 45088768 -> 46137344 00002000:00010000:0.0:1789762671.436654:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 88075 -> 90123 00010000:00010000:0.0:1789762671.436664:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02228 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.436673:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02228 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.436681:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02228 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.436686:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762671.436690:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02228 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.436704:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02228 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.436731:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02228 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.436737:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02228 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762671.436807:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81380a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014839552:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14239us (195674us total) trans 4294967497 rc 0/0 00000100:00100000:0.0:1789762671.436815:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 79 00000100:00100000:0.0:1789762671.436822:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194726364439 time 194726440284 state 0x2 ref 0 00000100:00100000:2.0:1789762671.439454:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014839680 00000100:00100000:2.0:1789762671.439462:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014839680 00000100:00100000:2.0:1789762671.439495:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194726364439 time 194726440284 state 0x0 ref 1 00000100:00100000:2.0:1789762671.439504:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194926440284 time 194726440284 state 0x1 ref 1 00000100:00100000:0.0:1789762671.622641:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194926440284 time 194926543524 state 0x0 ref 1 00000100:00100000:0.0:1789762671.622670:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014839680:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762671.622722:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 46137344, mtime: 0x6aad9c6f, atime: 0x6aad9c66, ctime: 0x6aad9c6f, blocks: 0x1600b 00010000:00010000:0.0:1789762671.622730:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0222f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.622743:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=194s v=(0 0) 00010000:00010000:0.0:1789762671.622747:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762671.622751:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0222f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.622788:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0222f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762671.634451:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 46137344 -> 47185920 00002000:00010000:0.0:1789762671.634455:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 90123 -> 92171 00010000:00010000:0.0:1789762671.634466:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0222f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.634476:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0222f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.634482:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0222f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.634489:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762671.634494:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0222f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.634504:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0222f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.634524:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0222f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.634533:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0222f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762671.634608:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014839680:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11949us (195197us total) trans 4294967498 rc 0/0 00000100:00100000:0.0:1789762671.634617:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 80 00000100:00100000:0.0:1789762671.634624:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194926440284 time 194926543524 state 0x2 ref 0 00000100:00100000:2.0:1789762671.636984:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762671.637032:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014839808 00000100:00100000:0.0:1789762671.637041:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014839808 00000100:00100000:0.0:1789762671.637076:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 194926440284 time 194926543524 state 0x0 ref 1 00000100:00100000:0.0:1789762671.637103:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195126543524 time 194926543524 state 0x1 ref 1 00000100:00100000:0.0:1789762671.822702:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195126543524 time 195126604531 state 0x0 ref 1 00000100:00100000:0.0:1789762671.822730:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81380000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014839808:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762671.822778:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 47185920, mtime: 0x6aad9c6f, atime: 0x6aad9c66, ctime: 0x6aad9c6f, blocks: 0x1680b 00010000:00010000:0.0:1789762671.822786:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02236 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.822799:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=195s v=(0 0) 00010000:00010000:0.0:1789762671.822803:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762671.822808:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.822829:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762671.833845:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967499 is committed 00002000:00010000:0.0:1789762671.833868:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 47185920 -> 48234496 00002000:00010000:0.0:1789762671.833874:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 92171 -> 94219 00010000:00010000:0.0:1789762671.833885:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02236 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.833894:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02236 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.833900:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02236 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.833906:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762671.833909:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02236 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.833916:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02236 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.833929:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02236 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762671.833934:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02236 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762671.833985:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81380000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014839808:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11268us (197003us total) trans 4294967499 rc 0/0 00000100:00100000:0.0:1789762671.834000:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 81 00000100:00100000:0.0:1789762671.834006:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195126543524 time 195126604531 state 0x2 ref 0 00000100:00100000:2.0:1789762671.836207:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762672.022921:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195326604531 time 195326819653 state 0x0 ref 1 00000100:00100000:0.0:1789762672.022959:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81381500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014839936:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762672.023031:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 48234496, mtime: 0x6aad9c6f, atime: 0x6aad9c66, ctime: 0x6aad9c6f, blocks: 0x1700b 00010000:00010000:0.0:1789762672.023038:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0223d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.023069:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=195s v=(0 0) 00010000:00010000:0.0:1789762672.023074:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762672.023080:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0223d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.023092:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0223d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762672.038316:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967500 is committed 00002000:00010000:0.0:1789762672.038406:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 48234496 -> 49283072 00002000:00010000:0.0:1789762672.038412:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb mtime from disk: 1789762671 -> 1789762672 00002000:00010000:0.0:1789762672.038414:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb ctime from disk: 1789762671 -> 1789762672 00002000:00010000:0.0:1789762672.038417:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 94219 -> 96267 00010000:00010000:0.0:1789762672.038428:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0223d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.038443:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0223d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.038465:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0223d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.038475:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762672.038482:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0223d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.038495:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0223d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.038516:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0223d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.038523:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0223d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762672.038594:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81381500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014839936:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15653us (202385us total) trans 4294967500 rc 0/0 00000100:00100000:0.0:1789762672.038604:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 82 00000100:00100000:0.0:1789762672.038611:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195326604531 time 195326819653 state 0x2 ref 0 00000100:00100000:0.0:1789762672.041556:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014840064 00000100:00100000:0.0:1789762672.041566:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014840064 00000100:00100000:0.0:1789762672.041605:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195326604531 time 195326819653 state 0x0 ref 1 00000100:00100000:0.0:1789762672.041616:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195526819653 time 195326819653 state 0x1 ref 1 00000100:00100000:0.0:1789762672.222998:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195526819653 time 195526898398 state 0x0 ref 1 00000100:00100000:0.0:1789762672.223034:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014840064:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762672.223113:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 49283072, mtime: 0x6aad9c70, atime: 0x6aad9c66, ctime: 0x6aad9c70, blocks: 0x1780b 00010000:00010000:0.0:1789762672.223121:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02244 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.223135:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=195s v=(0 0) 00010000:00010000:0.0:1789762672.223139:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762672.223144:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.223155:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762672.233946:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 49283072 -> 50331648 00002000:00010000:0.0:1789762672.233951:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 96267 -> 98315 00010000:00010000:0.0:1789762672.233961:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02244 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.233970:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02244 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.233977:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02244 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.233998:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762672.234003:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02244 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.234014:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02244 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.234033:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02244 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.234040:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02244 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762672.234119:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014840064:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11102us (192619us total) trans 4294967501 rc 0/0 00000100:00100000:0.0:1789762672.234128:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 83 00000100:00100000:0.0:1789762672.234134:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195526819653 time 195526898398 state 0x2 ref 0 00000100:00100000:2.0:1789762672.236222:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014840192 00000100:00100000:2.0:1789762672.236230:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014840192 00000100:00100000:2.0:1789762672.236264:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195526819653 time 195526898398 state 0x0 ref 1 00000100:00100000:2.0:1789762672.236273:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195726898398 time 195526898398 state 0x1 ref 1 00000100:00100000:0.0:1789762672.423118:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195726898398 time 195726996376 state 0x0 ref 1 00000100:00100000:0.0:1789762672.423141:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014840192:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762672.423190:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 50331648, mtime: 0x6aad9c70, atime: 0x6aad9c66, ctime: 0x6aad9c70, blocks: 0x1800b 00010000:00010000:0.0:1789762672.423197:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0224b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.423225:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=195s v=(0 0) 00010000:00010000:0.0:1789762672.423228:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762672.423233:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0224b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.423243:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0224b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762672.433957:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967502 is committed 00002000:00010000:0.0:1789762672.433974:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 50331648 -> 51380224 00002000:00010000:0.0:1789762672.433981:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 98315 -> 100363 00010000:00010000:0.0:1789762672.433993:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0224b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.434006:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0224b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.434017:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0224b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.434027:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762672.434030:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0224b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.434038:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0224b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.434064:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0224b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.434069:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b0224b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762672.434160:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014840192:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11025us (198018us total) trans 4294967502 rc 0/0 00000100:00100000:0.0:1789762672.434168:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 84 00000100:00100000:0.0:1789762672.434173:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195726898398 time 195726996376 state 0x2 ref 0 00000100:00100000:0.0:1789762672.436444:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014840320 00000100:00100000:0.0:1789762672.436452:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014840320 00000100:00100000:0.0:1789762672.436484:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195726898398 time 195726996376 state 0x0 ref 1 00000100:00100000:0.0:1789762672.436494:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195926996376 time 195726996376 state 0x1 ref 1 00000100:00100000:0.0:1789762672.623161:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195926996376 time 195927064626 state 0x0 ref 1 00000100:00100000:0.0:1789762672.623190:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014840320:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762672.623240:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 51380224, mtime: 0x6aad9c70, atime: 0x6aad9c66, ctime: 0x6aad9c70, blocks: 0x1880b 00010000:00010000:0.0:1789762672.623248:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02252 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.623261:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=195s v=(0 0) 00010000:00010000:0.0:1789762672.623265:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762672.623282:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.623293:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762672.636454:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967503 is committed 00002000:00010000:0.0:1789762672.636486:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 51380224 -> 52428800 00002000:00010000:0.0:1789762672.636492:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 100363 -> 102411 00010000:00010000:0.0:1789762672.636502:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02252 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.636513:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02252 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.636520:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02252 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.636526:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762672.636530:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02252 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.636538:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02252 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.636553:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02252 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.636571:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02252 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762672.636635:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014840320:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13456us (200223us total) trans 4294967503 rc 0/0 00000100:00100000:0.0:1789762672.636644:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 85 00000100:00100000:0.0:1789762672.636650:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 195926996376 time 195927064626 state 0x2 ref 0 00000100:00100000:2.0:1789762672.639327:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762672.823250:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196127064626 time 196127150109 state 0x0 ref 1 00000100:00100000:0.0:1789762672.823299:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81382300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014840448:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762672.823406:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 52428800, mtime: 0x6aad9c70, atime: 0x6aad9c66, ctime: 0x6aad9c70, blocks: 0x1900b 00010000:00010000:0.0:1789762672.823416:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02259 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.823450:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=196s v=(0 0) 00010000:00010000:0.0:1789762672.823457:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762672.823465:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762672.823490:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762672.844611:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 52428800 -> 53477376 00002000:00010000:2.0:1789762672.844617:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 102411 -> 104459 00010000:00010000:2.0:1789762672.844631:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02259 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.844643:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02259 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.844651:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02259 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.844658:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762672.844662:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02259 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.844670:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02259 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.844698:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02259 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762672.844705:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b02259 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762672.844789:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81382300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014840448:12345-192.168.203.48@tcp:4:dd.500 Request processed in 21510us (205470us total) trans 4294967504 rc 0/0 00000100:00100000:2.0:1789762672.844820:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 86 00000100:00100000:2.0:1789762672.844832:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196127064626 time 196127150109 state 0x2 ref 0 00000100:00100000:0.0:1789762672.848370:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014840576 00000100:00100000:0.0:1789762672.848381:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014840576 00000100:00100000:0.0:1789762672.848430:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196127064626 time 196127150109 state 0x0 ref 1 00000100:00100000:0.0:1789762672.848443:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196327150109 time 196127150109 state 0x1 ref 1 00000100:00100000:0.0:1789762673.023341:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196327150109 time 196327240269 state 0x0 ref 1 00000100:00100000:0.0:1789762673.023406:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014840576:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:2.0:1789762673.023444:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789762673.023472:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 53477376, mtime: 0x6aad9c70, atime: 0x6aad9c66, ctime: 0x6aad9c70, blocks: 0x1980b 00010000:00010000:0.0:1789762673.023485:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02260 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.023505:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=196s v=(0 0) 00010000:00010000:0.0:1789762673.023512:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762673.023521:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.023539:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762673.038720:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:0.0:1789762673.038725:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb mtime from disk: 1789762672 -> 1789762673 00002000:00010000:0.0:1789762673.038727:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb ctime from disk: 1789762672 -> 1789762673 00002000:00010000:0.0:1789762673.038730:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 104459 -> 106507 00010000:00010000:0.0:1789762673.038742:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02260 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.038754:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02260 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.038769:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02260 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.038775:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762673.038779:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02260 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.038787:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02260 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.038803:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02260 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.038810:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02260 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762673.038895:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014840576:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15520us (190580us total) trans 4294967505 rc 0/0 00000100:00100000:0.0:1789762673.038905:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 87 00000100:00100000:0.0:1789762673.038913:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196327150109 time 196327240269 state 0x2 ref 0 00000100:00100000:2.0:1789762673.041656:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014840704 00000100:00100000:2.0:1789762673.041666:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014840704 00000100:00100000:2.0:1789762673.041705:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196327150109 time 196327240269 state 0x0 ref 1 00000100:00100000:2.0:1789762673.041716:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196527240269 time 196327240269 state 0x1 ref 1 00000100:00100000:2.0:1789762673.223461:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196527240269 time 196527356203 state 0x0 ref 1 00000100:00100000:2.0:1789762673.223510:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014840704:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762673.223609:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 54525952, mtime: 0x6aad9c71, atime: 0x6aad9c66, ctime: 0x6aad9c71, blocks: 0x1a00b 00010000:00010000:2.0:1789762673.223623:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e2045fd5/0x416193e012b02267 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762673.223643:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=196s v=(0 0) 00010000:00010000:2.0:1789762673.223671:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762673.223683:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e2045fd5/0x416193e012b02267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762673.223708:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e2045fd5/0x416193e012b02267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762673.244535:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967506 is committed 00002000:00010000:0.0:1789762673.244591:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 54525952 -> 55574528 00002000:00010000:0.0:1789762673.244600:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 106507 -> 108555 00010000:00010000:0.0:1789762673.244614:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e2045fd5/0x416193e012b02267 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.244648:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e2045fd5/0x416193e012b02267 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.244658:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e2045fd5/0x416193e012b02267 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.244666:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebac31200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762673.244672:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e2045fd5/0x416193e012b02267 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.244682:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e2045fd5/0x416193e012b02267 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.244703:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e2045fd5/0x416193e012b02267 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.244717:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e2045fd5/0x416193e012b02267 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762673.244834:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014840704:12345-192.168.203.48@tcp:4:dd.500 Request processed in 21344us (203234us total) trans 4294967506 rc 0/0 00000100:00100000:0.0:1789762673.244848:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 88 00000100:00100000:0.0:1789762673.244857:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196527240269 time 196527356203 state 0x2 ref 0 00000100:00100000:0.0:1789762673.248589:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014840832 00000100:00100000:0.0:1789762673.248609:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014840832 00000100:00100000:0.0:1789762673.248654:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196527240269 time 196527356203 state 0x0 ref 1 00000100:00100000:0.0:1789762673.248667:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196727356203 time 196527356203 state 0x1 ref 1 00000100:00100000:0.0:1789762673.423581:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196727356203 time 196727479244 state 0x0 ref 1 00000100:00100000:0.0:1789762673.423634:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014840832:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762673.423709:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 55574528, mtime: 0x6aad9c71, atime: 0x6aad9c66, ctime: 0x6aad9c71, blocks: 0x1a80b 00010000:00010000:0.0:1789762673.423726:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0226e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.423753:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=196s v=(0 0) 00010000:00010000:0.0:1789762673.423763:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762673.423776:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0226e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.423819:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0226e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762673.443635:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967507 is committed 00002000:00010000:0.0:1789762673.443668:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 55574528 -> 56623104 00002000:00010000:0.0:1789762673.443675:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 108555 -> 110603 00010000:00010000:0.0:1789762673.443690:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0226e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.443709:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0226e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.443743:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0226e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.443756:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762673.443764:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0226e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.443779:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0226e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.443806:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0226e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.443820:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0226e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762673.443916:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014840832:12345-192.168.203.48@tcp:4:dd.500 Request processed in 20310us (195378us total) trans 4294967507 rc 0/0 00000100:00100000:0.0:1789762673.443929:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 89 00000100:00100000:0.0:1789762673.443937:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196727356203 time 196727479244 state 0x2 ref 0 00000100:00100000:2.0:1789762673.447404:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014840960 00000100:00100000:2.0:1789762673.447414:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014840960 00000100:00100000:2.0:1789762673.447458:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196727356203 time 196727479244 state 0x0 ref 1 00000100:00100000:2.0:1789762673.447470:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196927479244 time 196727479244 state 0x1 ref 1 00000100:00100000:0.0:1789762673.623686:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196927479244 time 196927591376 state 0x0 ref 1 00000100:00100000:0.0:1789762673.623712:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e18000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014840960:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762673.623761:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 56623104, mtime: 0x6aad9c71, atime: 0x6aad9c66, ctime: 0x6aad9c71, blocks: 0x1b00b 00010000:00010000:0.0:1789762673.623781:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02275 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.623794:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=196s v=(0 0) 00010000:00010000:0.0:1789762673.623798:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762673.623803:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.623813:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762673.634433:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967508 is committed 00002000:00010000:0.0:1789762673.634461:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:0.0:1789762673.634466:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 110603 -> 112651 00010000:00010000:0.0:1789762673.634476:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02275 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.634486:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02275 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.634493:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02275 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.634498:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762673.634501:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02275 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.634508:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02275 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.634531:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02275 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.634537:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02275 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762673.634603:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e18000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014840960:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10901us (187273us total) trans 4294967508 rc 0/0 00000100:00100000:0.0:1789762673.634611:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 90 00000100:00100000:0.0:1789762673.634618:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196927479244 time 196927591376 state 0x2 ref 0 00000100:00100000:2.0:1789762673.637129:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762673.637166:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014841088 00000100:00100000:0.0:1789762673.637175:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014841088 00000100:00100000:0.0:1789762673.637208:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 196927479244 time 196927591376 state 0x0 ref 1 00000100:00100000:0.0:1789762673.637218:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197127591376 time 196927591376 state 0x1 ref 1 00000100:00100000:0.0:1789762673.823753:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197127591376 time 197127655686 state 0x0 ref 1 00000100:00100000:0.0:1789762673.823781:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81383480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014841088:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762673.823831:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 57671680, mtime: 0x6aad9c71, atime: 0x6aad9c66, ctime: 0x6aad9c71, blocks: 0x1b80b 00010000:00010000:0.0:1789762673.823839:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0227c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.823852:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=197s v=(0 0) 00010000:00010000:0.0:1789762673.823856:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762673.823873:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0227c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.823886:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0227c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762673.835985:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967509 is committed 00002000:00010000:0.0:1789762673.836003:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:0.0:1789762673.836010:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 112651 -> 114699 00010000:00010000:0.0:1789762673.836024:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0227c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.836038:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0227c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.836049:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0227c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.836054:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762673.836058:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0227c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.836065:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0227c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.836080:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0227c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762673.836115:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0227c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762673.836188:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81383480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014841088:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12417us (199057us total) trans 4294967509 rc 0/0 00000100:00100000:0.0:1789762673.836196:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 91 00000100:00100000:0.0:1789762673.836203:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197127591376 time 197127655686 state 0x2 ref 0 00000100:00100000:2.0:1789762673.839362:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014841216 00000100:00100000:2.0:1789762673.839379:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014841216 00000100:00100000:2.0:1789762673.839415:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197127591376 time 197127655686 state 0x0 ref 1 00000100:00100000:2.0:1789762673.839425:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197327655686 time 197127655686 state 0x1 ref 1 00000100:00100000:0.0:1789762674.023846:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197327655686 time 197327749663 state 0x0 ref 1 00000100:00100000:0.0:1789762674.023873:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e1a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014841216:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762674.023923:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 58720256, mtime: 0x6aad9c71, atime: 0x6aad9c66, ctime: 0x6aad9c71, blocks: 0x1c00b 00010000:00010000:0.0:1789762674.023930:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02283 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.023943:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=197s v=(0 0) 00010000:00010000:0.0:1789762674.023947:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762674.023951:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.023962:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762674.035980:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:0.0:1789762674.035985:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb mtime from disk: 1789762673 -> 1789762674 00002000:00010000:0.0:1789762674.035994:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb ctime from disk: 1789762673 -> 1789762674 00002000:00010000:0.0:1789762674.035996:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x8:0x0] updating lvb blocks from disk: 114699 -> 116747 00010000:00010000:0.0:1789762674.036005:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02283 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.036013:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02283 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.036020:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02283 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.036025:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762674.036028:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02283 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.036035:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02283 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.036048:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02283 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.036053:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b02283 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762674.036137:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e1a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014841216:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12275us (196816us total) trans 4294967510 rc 0/0 00000100:00100000:0.0:1789762674.036144:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 92 00000100:00100000:0.0:1789762674.036154:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197327655686 time 197327749663 state 0x2 ref 0 00000100:00100000:0.0:1789762674.038522:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014841344 00000100:00100000:0.0:1789762674.038531:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014841344 00000100:00100000:0.0:1789762674.038562:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197327655686 time 197327749663 state 0x0 ref 1 00000100:00100000:0.0:1789762674.038571:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197527749663 time 197327749663 state 0x1 ref 1 00000100:00100000:0.0:1789762674.223930:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197527749663 time 197527832248 state 0x0 ref 1 00000100:00100000:0.0:1789762674.223957:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc426a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014841344:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762674.224006:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 59768832, mtime: 0x6aad9c72, atime: 0x6aad9c66, ctime: 0x6aad9c72, blocks: 0x1c80b 00010000:00010000:0.0:1789762674.224013:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0228a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.224026:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=197s v=(0 0) 00010000:00010000:0.0:1789762674.224030:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762674.224035:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0228a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.224045:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0228a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762674.237167:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967511 is committed 00000100:00100000:2.0:1789762674.239768:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014841472 00000100:00100000:2.0:1789762674.239778:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014841472 00000100:00100000:2.0:1789762674.239819:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197527749663 time 197527832248 state 0x0 ref 1 00000100:00100000:2.0:1789762674.239833:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197727832248 time 197527832248 state 0x1 ref 1 00000100:00100000:0.0:1789762674.424011:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197727832248 time 197727911079 state 0x0 ref 1 00000100:00100000:0.0:1789762674.424037:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014841472:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762674.424112:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 60817408, mtime: 0x6aad9c72, atime: 0x6aad9c66, ctime: 0x6aad9c72, blocks: 0x1d00b 00010000:00010000:0.0:1789762674.424134:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008743d688/0x416193e012b02291 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.424147:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=197s v=(0 0) 00010000:00010000:0.0:1789762674.424150:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762674.424155:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008743d688/0x416193e012b02291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762674.424166:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008743d688/0x416193e012b02291 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762674.435439:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967512 is committed 00000100:00100000:2.0:1789762674.437625:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762674.437675:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014841600 00000100:00100000:0.0:1789762674.437683:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014841600 00000100:00100000:0.0:1789762674.437722:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197727832248 time 197727911079 state 0x0 ref 1 00000100:00100000:0.0:1789762674.437734:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197927911079 time 197727911079 state 0x1 ref 1 00010000:00010000:0.0:1789762674.615190:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762674.636033:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967513 is committed 00000100:00100000:2.0:1789762674.638465:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014841728 00000100:00100000:2.0:1789762674.638473:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014841728 00000100:00100000:2.0:1789762674.638509:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 197927911079 time 197928028803 state 0x0 ref 1 00000100:00100000:2.0:1789762674.638519:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 198128028803 time 197928028803 state 0x1 ref 1 00000100:00100000:2.0:1789762674.786218:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762674.786255:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762674.786276:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014841984 00000100:00100000:2.0:1789762674.786278:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762674.786286:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014841984 00000100:00100000:2.0:1789762674.786322:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762674.786354:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762674.786371:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014842368 00000100:00100000:0.0:1789762674.786378:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014842368 00000100:00080000:2.0:1789762674.786385:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762674.786394:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762674.786421:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 113 00000100:00100000:2.0:1789762674.786430:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 60 00000100:00100000:0.0:1789762674.786438:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81381c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014841984:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762674.786444:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81381f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014842240:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762674.786490:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81381c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014841984:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 55us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762674.786496:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81381f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014842240:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 55us (214us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762674.786509:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 113 00000100:00100000:2.0:1789762674.786514:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 60 00000100:00100000:0.0:1789762674.786540:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 114 00000100:00100000:0.0:1789762674.786564:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81380380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014842368:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762674.786597:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81380380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014842368:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 37us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762674.786606:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 114 00000100:00100000:2.0:1789762674.807231:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762674.807247:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81383b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022134144:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762674.807294:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789762674.807406:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762674.807576:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81383b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022134144:0@lo:13:osp-pre-0-0.0 00000001:00080000:2.0:1789762674.816019:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967344 is committed 00000001:00080000:2.0:1789762674.835313:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967514 is committed 00000100:00100000:2.0:1789762674.837483:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762674.837537:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014842496 00000100:00100000:0.0:1789762674.837545:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014842496 00000100:00100000:0.0:1789762674.837578:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 198128028803 time 198128116233 state 0x0 ref 1 00000100:00100000:0.0:1789762674.837587:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 198328116233 time 198128116233 state 0x1 ref 1 00000100:00080000:0.0:1789762675.024388:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789762675.035359:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967515 is committed 00000100:00100000:2.0:1789762675.037542:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762675.236664:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967516 is committed 00000100:00100000:2.0:1789762675.238948:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762675.239001:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014842752 00000100:00100000:0.0:1789762675.239009:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014842752 00000100:00100000:0.0:1789762675.239051:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 198528228699 time 198528325248 state 0x0 ref 1 00000100:00100000:0.0:1789762675.239065:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 198728325248 time 198528325248 state 0x1 ref 1 00000100:00100000:2.0:1789762675.319350:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81383100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022134656:0@lo:400:kworker.0 00000100:00100000:2.0:1789762675.319404:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebf51d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022134272:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762675.319439:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762675.319465:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e8600ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022134912:0@lo:400:kworker.0 00000100:00100000:2.0:1789762675.319502:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762675.319563:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022134528 00000100:00100000:2.0:1789762675.319569:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022134528 00000100:00100000:0.0:1789762675.319574:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022134912 00000100:00100000:0.0:1789762675.319581:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022134912 00000100:00100000:2.0:1789762675.319623:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022134784 00000100:00100000:2.0:1789762675.319629:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022134784 00000100:00100000:0.0:1789762675.319652:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 62 00000100:00100000:0.0:1789762675.319674:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022134912:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762675.319678:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 116 00000100:00100000:2.0:1789762675.319695:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022134656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762675.319753:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022134912:12345-0@lo:400:kworker.0 Request processed in 88us (250us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762675.319763:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022134656:12345-0@lo:400:kworker.0 Request processed in 70us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762675.319765:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 62 00000100:00100000:2.0:1789762675.319779:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 116 00000100:00100000:2.0:1789762675.319826:0:3274:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967344 gen 2 00000004:00080000:2.0:1789762675.319833:0:3274:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebfdf9f80, transno 4294967344 00000100:00100000:2.0:1789762675.319854:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e8600ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022134912:0@lo:400:kworker.0 00000100:00100000:2.0:1789762675.319906:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81383100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022134656:0@lo:400:kworker.0 00000100:00100000:2.0:1789762675.319953:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022134272 00000100:00100000:2.0:1789762675.319957:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022134272 00000100:00100000:2.0:1789762675.320000:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:2.0:1789762675.320010:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022134272:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762675.320038:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762675.320123:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022134272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 113us (682us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762675.320130:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000001:00080000:2.0:1789762675.435739:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967517 is committed 00000100:00100000:2.0:1789762675.437999:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762675.637829:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967518 is committed 00000100:00100000:2.0:1789762675.640110:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762675.837392:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967519 is committed 00000100:00100000:2.0:1789762675.839981:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762676.037809:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967520 is committed 00000100:00100000:2.0:1789762676.040368:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014843264 00000100:00100000:2.0:1789762676.040378:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014843264 00000100:00100000:2.0:1789762676.040425:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 199328619163 time 199328703513 state 0x0 ref 1 00000100:00100000:2.0:1789762676.040436:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 199528703513 time 199328703513 state 0x1 ref 1 00000001:00080000:2.0:1789762676.237114:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967521 is committed 00000100:00100000:2.0:1789762676.239754:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014843392 00000100:00100000:2.0:1789762676.239764:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014843392 00000100:00100000:2.0:1789762676.239809:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 199528703513 time 199528789656 state 0x0 ref 1 00000100:00100000:2.0:1789762676.239824:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 199728789656 time 199528789656 state 0x1 ref 1 00000001:00080000:2.0:1789762676.439560:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967522 is committed 00000100:00100000:2.0:1789762676.442469:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014843520 00000100:00100000:2.0:1789762676.442479:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014843520 00000100:00100000:2.0:1789762676.442538:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 199728789656 time 199728906114 state 0x0 ref 1 00000100:00100000:2.0:1789762676.442549:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 199928906114 time 199728906114 state 0x1 ref 1 00000100:00100000:2.0:1789762676.644314:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014843648 00000100:00100000:2.0:1789762676.644362:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014843648 00000100:00100000:2.0:1789762676.644435:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 199928906114 time 199929108035 state 0x0 ref 1 00000100:00100000:2.0:1789762676.644455:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 200129108035 time 199929108035 state 0x1 ref 1 00000001:00080000:2.0:1789762676.838363:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967524 is committed 00000100:00100000:2.0:1789762676.840808:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014843776 00000100:00100000:2.0:1789762676.840819:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014843776 00000100:00100000:2.0:1789762676.840873:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 200129108035 time 200129243906 state 0x0 ref 1 00000100:00100000:2.0:1789762676.840890:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 200329243906 time 200129243906 state 0x1 ref 1 00000100:00080000:0.0:1789762677.025505:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789762677.038455:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967525 is committed 00000100:00100000:2.0:1789762677.040823:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014843904 00000100:00100000:2.0:1789762677.040834:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014843904 00000100:00100000:2.0:1789762677.040891:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 200329243906 time 200329326930 state 0x0 ref 1 00000100:00100000:2.0:1789762677.040907:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 200529326930 time 200329326930 state 0x1 ref 1 00000100:00100000:2.0:1789762677.241399:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014844032 00000100:00100000:2.0:1789762677.241414:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014844032 00000100:00100000:2.0:1789762677.241473:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 200529326930 time 200529440553 state 0x0 ref 1 00000100:00100000:2.0:1789762677.241489:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 200729440553 time 200529440553 state 0x1 ref 1 00000001:00080000:2.0:1789762677.449835:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967527 is committed 00000100:00100000:2.0:1789762677.454578:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762677.643996:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967528 is committed 00000100:00100000:2.0:1789762677.648205:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014844288 00000100:00100000:2.0:1789762677.648216:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014844288 00000100:00100000:2.0:1789762677.648264:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 200929588715 time 200929795686 state 0x0 ref 1 00000100:00100000:2.0:1789762677.648294:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 201129795686 time 200929795686 state 0x1 ref 1 00000100:00100000:2.0:1789762677.849879:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762678.039549:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014844544 00000100:00100000:2.0:1789762678.039557:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014844544 00000100:00100000:2.0:1789762678.039592:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 201329946961 time 201330037900 state 0x0 ref 1 00000100:00100000:2.0:1789762678.039602:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 201530037900 time 201330037900 state 0x1 ref 1 00000001:00080000:0.0:1789762678.237921:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967531 is committed 00000100:00100000:2.0:1789762678.240357:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014844672 00000100:00100000:2.0:1789762678.240365:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014844672 00000100:00100000:2.0:1789762678.240396:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 201530037900 time 201530117558 state 0x0 ref 1 00000100:00100000:2.0:1789762678.240405:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 201730117558 time 201530117558 state 0x1 ref 1 00000001:00080000:0.0:1789762678.437390:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967532 is committed 00000100:00100000:2.0:1789762678.439979:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014844800 00000100:00100000:2.0:1789762678.439988:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014844800 00000100:00100000:2.0:1789762678.440018:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 201730117558 time 201730242171 state 0x0 ref 1 00000100:00100000:2.0:1789762678.440027:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 201930242171 time 201730242171 state 0x1 ref 1 00000001:00080000:0.0:1789762678.637820:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967533 is committed 00000100:00100000:2.0:1789762678.640446:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014844928 00000100:00100000:2.0:1789762678.640454:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014844928 00000100:00100000:2.0:1789762678.640488:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 201930242171 time 201930314555 state 0x0 ref 1 00000100:00100000:2.0:1789762678.640498:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 202130314555 time 201930314555 state 0x1 ref 1 00010000:00010000:0.0:1789762678.711174:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762678.838043:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967534 is committed 00000100:00100000:2.0:1789762678.840958:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014845056 00000100:00100000:2.0:1789762678.840967:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014845056 00000100:00100000:2.0:1789762678.841004:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 202130314555 time 202130403368 state 0x0 ref 1 00000100:00100000:2.0:1789762678.841013:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 202330403368 time 202130403368 state 0x1 ref 1 00000100:00080000:0.0:1789762679.026696:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789762679.037624:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967535 is committed 00000100:00100000:2.0:1789762679.040077:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014845184 00000100:00100000:2.0:1789762679.040106:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014845184 00000100:00100000:2.0:1789762679.040139:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 202330403368 time 202330497115 state 0x0 ref 1 00000100:00100000:2.0:1789762679.040148:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 202530497115 time 202330497115 state 0x1 ref 1 00000001:00080000:0.0:1789762679.238739:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967536 is committed 00000100:00100000:2.0:1789762679.241059:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014845312 00000100:00100000:2.0:1789762679.241068:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014845312 00000100:00100000:2.0:1789762679.241118:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 202530497115 time 202530613190 state 0x0 ref 1 00000100:00100000:2.0:1789762679.241128:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 202730613190 time 202530613190 state 0x1 ref 1 00000100:00100000:2.0:1789762679.439656:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014845440 00000100:00100000:2.0:1789762679.439664:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014845440 00000100:00100000:2.0:1789762679.439694:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 202730613190 time 202730713043 state 0x0 ref 1 00000100:00100000:2.0:1789762679.439703:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 202930713043 time 202730713043 state 0x1 ref 1 00000001:00080000:0.0:1789762679.638885:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967538 is committed 00000100:00100000:2.0:1789762679.642491:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762679.838987:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967539 is committed 00000100:00100000:2.0:1789762679.841608:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014845696 00000100:00100000:2.0:1789762679.841616:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014845696 00000100:00100000:2.0:1789762679.841651:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 203130798893 time 203130908690 state 0x0 ref 1 00000100:00100000:2.0:1789762679.841661:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 203330908690 time 203130908690 state 0x1 ref 1 00000100:00100000:2.0:1789762679.906250:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762679.906302:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762679.906321:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762679.906349:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762679.906350:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014845952 00000100:00100000:0.0:1789762679.906363:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014845952 00000100:00100000:2.0:1789762679.906377:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014846080 00000100:00100000:2.0:1789762679.906385:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014846080 00000100:00100000:2.0:1789762679.906437:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 63 00000100:00100000:0.0:1789762679.906442:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 118 00000100:00100000:2.0:1789762679.906453:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014846080:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762679.906459:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014845952:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762679.906542:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014846080:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 93us (189us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762679.906548:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014845952:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 91us (301us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762679.906563:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 63 00000100:00100000:0.0:1789762679.906574:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 118 00000100:00080000:0.0:1789762679.906606:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762679.906613:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762679.927359:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022135040 00000100:00100000:0.0:1789762679.927368:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022135040 00000100:00100000:0.0:1789762679.927437:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 53 00000100:00100000:0.0:1789762679.927451:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022135040:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762679.927469:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 218038272 pending 0 free 3775922176 avail 3767533568 00000100:00100000:0.0:1789762679.927521:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022135040:12345-0@lo:13:osp-pre-0-0.0 Request processed in 74us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762679.927531:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 53 00000001:00080000:0.0:1789762680.038807:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967540 is committed 00000100:00100000:2.0:1789762680.041379:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014846336 00000100:00100000:2.0:1789762680.041387:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014846336 00000100:00100000:2.0:1789762680.041420:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 203330908690 time 203330995653 state 0x0 ref 1 00000100:00100000:2.0:1789762680.041429:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 203530995653 time 203330995653 state 0x1 ref 1 00000100:00100000:2.0:1789762680.240986:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014846464 00000100:00100000:2.0:1789762680.240993:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014846464 00000100:00100000:2.0:1789762680.241026:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 203530995653 time 203531083481 state 0x0 ref 1 00000100:00100000:2.0:1789762680.241035:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 203731083481 time 203531083481 state 0x1 ref 1 00000100:00100000:2.0:1789762680.439296:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022135296 00000100:00100000:2.0:1789762680.439304:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022135296 00000100:00100000:2.0:1789762680.439364:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 54 00000100:00100000:2.0:1789762680.439377:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022135296:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762680.439406:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762680.439465:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022135296:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762680.439475:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 54 00000100:00080000:2.0:1789762680.439625:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762680.439664:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022135552 00000100:00100000:2.0:1789762680.439669:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022135552 00000100:00100000:2.0:1789762680.439710:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00000100:00100000:2.0:1789762680.439722:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022135552:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762680.439729:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e8600fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022135680:0@lo:400:kworker.0 00000100:00100000:2.0:1789762680.439792:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022135552:12345-0@lo:400:kworker.0 Request processed in 74us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762680.439806:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000100:00100000:0.0:1789762680.439810:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e8600c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022135552:0@lo:400:kworker.0 00000100:00080000:2.0:1789762680.439830:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762680.439937:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e8600df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022135168:0@lo:400:kworker.0 00000100:00100000:2.0:1789762680.440074:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762680.440121:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014846592 00000100:00100000:2.0:1789762680.440124:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014846592 00000100:00100000:2.0:1789762680.440152:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 203731083481 time 203731153536 state 0x0 ref 1 00000100:00100000:2.0:1789762680.440161:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 203931153536 time 203731153536 state 0x1 ref 1 00000001:00080000:0.0:1789762680.639633:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967543 is committed 00000100:00100000:2.0:1789762680.642109:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014846720 00000100:00100000:2.0:1789762680.642118:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014846720 00000100:00100000:2.0:1789762680.642154:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 203931153536 time 203931268875 state 0x0 ref 1 00000100:00100000:2.0:1789762680.642163:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 204131268875 time 203931268875 state 0x1 ref 1 00000001:00080000:0.0:1789762680.838846:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967544 is committed 00000100:00100000:2.0:1789762680.841167:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762680.841224:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014846848 00000100:00100000:0.0:1789762680.841233:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014846848 00000100:00100000:0.0:1789762680.841282:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 204131268875 time 204131393440 state 0x0 ref 1 00000100:00100000:0.0:1789762680.841291:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 204331393440 time 204131393440 state 0x1 ref 1 00000100:00080000:2.0:1789762681.027669:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762681.041392:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762681.240326:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967546 is committed 00000100:00100000:0.0:1789762681.242685:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014847104 00000100:00100000:0.0:1789762681.242693:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014847104 00000100:00100000:0.0:1789762681.242731:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 204531500321 time 204531625128 state 0x0 ref 1 00000100:00100000:0.0:1789762681.242741:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 204731625128 time 204531625128 state 0x1 ref 1 00000001:00080000:0.0:1789762681.442113:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967547 is committed 00000100:00100000:2.0:1789762681.444745:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762681.641012:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967548 is committed 00000100:00100000:0.0:1789762681.644211:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014847360 00000100:00100000:0.0:1789762681.644222:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014847360 00000100:00100000:0.0:1789762681.644304:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 204931737956 time 204931849131 state 0x0 ref 1 00000100:00100000:0.0:1789762681.644321:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 205131849131 time 204931849131 state 0x1 ref 1 00000100:00100000:2.0:1789762681.843282:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014847488 00000100:00100000:2.0:1789762681.843293:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014847488 00000100:00100000:2.0:1789762681.843342:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 205131849131 time 205131958910 state 0x0 ref 1 00000100:00100000:2.0:1789762681.843355:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 205331958910 time 205131958910 state 0x1 ref 1 00000001:00080000:0.0:1789762682.040387:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967550 is committed 00000100:00100000:2.0:1789762682.043122:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762682.043175:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014847616 00000100:00100000:0.0:1789762682.043188:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014847616 00000100:00100000:0.0:1789762682.043238:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 205331958910 time 205332093655 state 0x0 ref 1 00000100:00100000:0.0:1789762682.043263:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 205532093655 time 205332093655 state 0x1 ref 1 00000001:00080000:0.0:1789762682.240226:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967551 is committed 00000100:00100000:2.0:1789762682.243108:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762682.442443:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762682.442490:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014847872 00000100:00100000:0.0:1789762682.442501:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014847872 00000100:00100000:0.0:1789762682.442544:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 205732218341 time 205732285073 state 0x0 ref 1 00000100:00100000:0.0:1789762682.442556:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 205932285073 time 205732285073 state 0x1 ref 1 00000100:00100000:2.0:1789762682.641458:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762682.641590:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762682.688272:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762682.691005:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762682.691050:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014848512 00000100:00100000:0.0:1789762682.691056:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014848512 00000100:00100000:2.0:1789762682.691117:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 90 00000100:00100000:2.0:1789762682.691128:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81382680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014848512:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:2.0:1789762682.691137:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762682.691150:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097f13257/0x416193e012b023da lrc: 2/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd226 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762682.691172:0:6648:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x8:0x0] initial LVB size: 104857600, mtime: 0x6aad9c7a, atime: 0x6aad9c66, ctime: 0x6aad9c7a, blocks: 0x3200f 00010000:00010000:2.0:1789762682.691186:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=205s v=(0 0) 00010000:00010000:2.0:1789762682.691190:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762682.691198:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097f13257/0x416193e012b023da lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd226 expref: 6 pid: 6648 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762682.691209:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000097f13257/0x416193e012b023da lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd226 expref: 6 pid: 6648 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762682.691219:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebac31c00, rc 0) 00000100:00100000:2.0:1789762682.691255:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81382680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014848512:12345-192.168.203.48@tcp:101:dd.500 Request processed in 130us (259us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762682.691260:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 90 00000100:00100000:2.0:1789762682.693452:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762682.807226:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762682.835389:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762682.835450:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014848768 00000100:00100000:0.0:1789762682.835462:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014848768 00000100:00100000:0.0:1789762682.835511:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 206132378073 time 206132435345 state 0x0 ref 1 00000100:00100000:0.0:1789762682.835526:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 206332435345 time 206132435345 state 0x1 ref 1 00000100:00080000:2.0:1789762683.028748:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762683.034297:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762683.234258:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762683.234313:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014849024 00000100:00100000:2.0:1789762683.234338:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014849024 00000100:00100000:2.0:1789762683.234388:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 206532566095 time 206532671778 state 0x0 ref 1 00000100:00100000:2.0:1789762683.234400:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 206732671778 time 206532671778 state 0x1 ref 1 00000100:00100000:0.0:1789762683.635576:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014849280 00000100:00100000:0.0:1789762683.635585:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014849280 00000100:00100000:0.0:1789762683.635627:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 206932794193 time 206932874123 state 0x0 ref 1 00000100:00100000:0.0:1789762683.635637:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 207132874123 time 206932874123 state 0x1 ref 1 00000100:00100000:2.0:1789762683.835372:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014849408 00000100:00100000:2.0:1789762683.835387:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014849408 00000100:00100000:2.0:1789762683.835439:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 207132874123 time 207132957337 state 0x0 ref 1 00000100:00100000:2.0:1789762683.835452:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 207332957337 time 207132957337 state 0x1 ref 1 00000100:00100000:0.0:1789762684.034477:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014849536 00000100:00100000:0.0:1789762684.034485:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014849536 00000100:00100000:0.0:1789762684.034515:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 207332957337 time 207333031345 state 0x0 ref 1 00000100:00100000:0.0:1789762684.034524:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 207533031345 time 207333031345 state 0x1 ref 1 00000100:00100000:2.0:1789762684.235215:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014849664 00000100:00100000:2.0:1789762684.235223:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014849664 00000100:00100000:2.0:1789762684.235272:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 207533031345 time 207533132028 state 0x0 ref 1 00000100:00100000:2.0:1789762684.235281:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 207733132028 time 207533132028 state 0x1 ref 1 00000100:00100000:0.0:1789762684.435319:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014849792 00000100:00100000:0.0:1789762684.435328:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014849792 00000100:00100000:0.0:1789762684.435360:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 207733132028 time 207733234852 state 0x0 ref 1 00000100:00100000:0.0:1789762684.435378:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 207933234852 time 207733234852 state 0x1 ref 1 00000100:00100000:2.0:1789762684.635120:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014849920 00000100:00100000:2.0:1789762684.635129:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014849920 00000100:00100000:2.0:1789762684.635162:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 207933234852 time 207933321415 state 0x0 ref 1 00000100:00100000:2.0:1789762684.635171:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 208133321415 time 207933321415 state 0x1 ref 1 00000100:00100000:0.0:1789762684.834683:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014850048 00000100:00100000:0.0:1789762684.834691:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014850048 00000100:00100000:0.0:1789762684.834731:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 208133321415 time 208133424985 state 0x0 ref 1 00000100:00100000:0.0:1789762684.834739:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 208333424985 time 208133424985 state 0x1 ref 1 00000100:00100000:2.0:1789762685.026316:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762685.026390:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762685.026445:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014850304 00000100:00100000:0.0:1789762685.026453:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014850304 00000100:00080000:2.0:1789762685.026504:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762685.026524:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 123 00000100:00100000:0.0:1789762685.026537:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc556d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2365:x1876702014850304:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762685.026596:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc556d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2365:x1876702014850304:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 52us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762685.026605:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 123 00000100:00100000:2.0:1789762685.028808:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014850432 00000100:00100000:2.0:1789762685.028814:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014850432 00000100:00100000:2.0:1789762685.028869:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 91 00000100:00080000:0.0:1789762685.028870:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762685.028877:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762685.028888:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82731880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2362:x1876702014850432:12345-192.168.203.48@tcp:9:kworker.0 00000020:00000080:2.0:1789762685.028915:0:10916:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x416193e012b01dac 00000100:00080000:2.0:1789762685.028934:0:10916:0:(import.c:65:import_set_state_nolock()) ffff981ebc4c9800 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1789762685.028976:0:10916:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967344 00000100:00100000:2.0:1789762685.029194:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82731880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+2:2362:x1876702014850432:12345-192.168.203.48@tcp:9:kworker.0 Request processed in 314us (433us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762685.029200:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 91 00000100:00080000:0.0:1789762685.029596:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762685.034458:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762685.047248:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc556a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022135936:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762685.047291:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789762685.047380:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00100000:2.0:1789762685.055459:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:2.0:1789762685.055463:0:7605:0:(genops.c:885:class_export_put()) final put ffff981ebc4c8000/f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:0.0:1789762685.055485:0:37:0:(genops.c:844:class_export_destroy()) destroying export ffff981ebc4c8000/f1bd28de-ee78-4d18-bad3-fa7cec968893 for lustre-OST0001 00000020:00000080:0.0:1789762685.055530:0:37:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff981ebc4c9800 for lustre-OST0001 00000100:00100000:0.0:1789762685.234992:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014850816 00000100:00100000:0.0:1789762685.235000:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014850816 00000100:00100000:0.0:1789762685.235033:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 208533475238 time 208533561077 state 0x0 ref 1 00000100:00100000:0.0:1789762685.235042:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 208733561077 time 208533561077 state 0x1 ref 1 00000100:00100000:0.0:1789762685.435832:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762685.559279:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022136320:0@lo:400:kworker.0 00000100:00100000:0.0:1789762685.559303:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc556680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022136192:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762685.559371:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762685.559372:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762685.559430:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022136576:0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.559437:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762685.559449:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022136448:0@lo:400:kworker.0 00000100:00100000:0.0:1789762685.559467:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762685.559495:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762685.559517:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022136320 00000100:00100000:0.0:1789762685.559525:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022136320 00000100:00100000:2.0:1789762685.559563:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022136192 00000100:00100000:0.0:1789762685.559569:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022136448 00000100:00100000:2.0:1789762685.559570:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022136192 00000100:00100000:0.0:1789762685.559574:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022136448 00000100:00100000:0.0:1789762685.559620:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 126 00000100:00100000:2.0:1789762685.559631:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 56 00000100:00100000:0.0:1789762685.559633:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022136576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.559646:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022136192:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762685.559678:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762685.559699:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022136576:12345-0@lo:400:kworker.0 Request processed in 70us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762685.559724:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 126 00000100:00100000:2.0:1789762685.559725:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022136192:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (358us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762685.559740:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 56 00000100:00100000:0.0:1789762685.559749:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:0.0:1789762685.559762:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc557100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022136448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.559813:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022136320:0@lo:400:kworker.0 00000100:00100000:0.0:1789762685.559817:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc557100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022136448:12345-0@lo:400:kworker.0 Request processed in 58us (321us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762685.559829:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00000100:00100000:2.0:1789762685.559849:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdffb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022136448:0@lo:400:kworker.0 00000100:00100000:0.0:1789762685.559880:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc556680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022136192:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762685.559900:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022136576:0@lo:400:kworker.0 00000100:00100000:2.0:1789762685.635917:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762685.635960:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014851072 00000100:00100000:0.0:1789762685.635968:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014851072 00000100:00100000:0.0:1789762685.635998:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 208933637440 time 208933707637 state 0x0 ref 1 00000100:00100000:0.0:1789762685.636007:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 209133707637 time 208933707637 state 0x1 ref 1 00000100:00100000:0.0:1789762685.835706:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762685.879191:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762686.036121:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014851328 00000100:00100000:0.0:1789762686.036131:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014851328 00000100:00100000:0.0:1789762686.036172:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 209333799553 time 209333861703 state 0x0 ref 1 00000100:00100000:0.0:1789762686.036182:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 209533861703 time 209333861703 state 0x1 ref 1 00000100:00100000:2.0:1789762686.235631:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762686.436175:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762686.436238:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014851584 00000100:00100000:2.0:1789762686.436256:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014851584 00000100:00100000:2.0:1789762686.436298:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 209733978313 time 209734048762 state 0x0 ref 1 00000100:00100000:2.0:1789762686.436309:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 209934048762 time 209734048762 state 0x1 ref 1 00000100:00100000:2.0:1789762686.837470:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014851840 00000100:00100000:2.0:1789762686.837479:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014851840 00000100:00100000:2.0:1789762686.837512:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 210134182465 time 210134279265 state 0x0 ref 1 00000100:00100000:2.0:1789762686.837520:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 210334279265 time 210134279265 state 0x1 ref 1 00000100:00080000:0.0:1789762687.030581:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762687.038921:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762687.238453:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762687.238502:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014852096 00000100:00100000:2.0:1789762687.238511:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014852096 00000100:00100000:2.0:1789762687.238550:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 210534363403 time 210534425878 state 0x0 ref 1 00000100:00100000:2.0:1789762687.238563:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 210734425878 time 210534425878 state 0x1 ref 1 00000100:00100000:0.0:1789762687.635924:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762687.635978:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014852352 00000100:00100000:2.0:1789762687.635987:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014852352 00000100:00100000:2.0:1789762687.636026:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 210934512788 time 210934558990 state 0x0 ref 1 00000100:00100000:2.0:1789762687.636036:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 211134558990 time 210934558990 state 0x1 ref 1 00000100:00100000:0.0:1789762687.836234:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762688.036167:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762688.036227:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014852608 00000100:00100000:0.0:1789762688.036241:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014852608 00000100:00100000:0.0:1789762688.036272:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 211334646316 time 211334716068 state 0x0 ref 1 00000100:00100000:0.0:1789762688.036281:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 211534716068 time 211334716068 state 0x1 ref 1 00000100:00100000:0.0:1789762688.437392:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014852864 00000100:00100000:0.0:1789762688.437403:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014852864 00000100:00100000:0.0:1789762688.437465:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 211734787731 time 211734873857 state 0x0 ref 1 00000100:00100000:0.0:1789762688.437474:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 211934873857 time 211734873857 state 0x1 ref 1 00000100:00100000:2.0:1789762688.636873:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014852992 00000100:00100000:2.0:1789762688.636881:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014852992 00000100:00100000:2.0:1789762688.636915:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 211934873857 time 211934994318 state 0x0 ref 1 00000100:00100000:2.0:1789762688.636925:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 212134994318 time 211934994318 state 0x1 ref 1 00000100:00100000:0.0:1789762688.836013:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762688.836061:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014853120 00000100:00100000:2.0:1789762688.836069:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014853120 00000100:00100000:2.0:1789762688.836116:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 212134994318 time 212135082678 state 0x0 ref 1 00000100:00100000:2.0:1789762688.836130:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 212335082678 time 212135082678 state 0x1 ref 1 00000100:00080000:0.0:1789762689.031350:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762689.036518:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762689.036566:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014853248 00000100:00100000:2.0:1789762689.036573:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014853248 00000100:00100000:2.0:1789762689.036602:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 212335082678 time 212335186552 state 0x0 ref 1 00000100:00100000:2.0:1789762689.036611:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 212535186552 time 212335186552 state 0x1 ref 1 00000100:00100000:2.0:1789762689.236884:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762689.236931:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014853376 00000100:00100000:0.0:1789762689.236947:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014853376 00000100:00100000:0.0:1789762689.236976:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 212535186552 time 212535279433 state 0x0 ref 1 00000100:00100000:0.0:1789762689.236985:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 212735279433 time 212535279433 state 0x1 ref 1 00000100:00100000:2.0:1789762689.437442:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762689.637069:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762689.637152:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014853632 00000100:00100000:2.0:1789762689.637160:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014853632 00000100:00100000:2.0:1789762689.637207:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 212935376855 time 212935466850 state 0x0 ref 1 00000100:00100000:2.0:1789762689.637215:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 213135466850 time 212935466850 state 0x1 ref 1 00000100:00100000:2.0:1789762689.837138:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014853760 00000100:00100000:2.0:1789762689.837148:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014853760 00000100:00100000:2.0:1789762689.837207:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 213135466850 time 213135590355 state 0x0 ref 1 00000100:00100000:2.0:1789762689.837222:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 213335590355 time 213135590355 state 0x1 ref 1 00010000:00010000:0.0:1789762689.975232:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762690.037100:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762690.037179:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014853888 00000100:00100000:0.0:1789762690.037210:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014853888 00000100:00100000:0.0:1789762690.037262:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 213335590355 time 213335691938 state 0x0 ref 1 00000100:00100000:0.0:1789762690.037274:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 213535691938 time 213335691938 state 0x1 ref 1 00000100:00100000:2.0:1789762690.146408:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014854016 00000100:00100000:2.0:1789762690.146415:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014854016 00000100:00100000:0.0:1789762690.146448:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 128 00000100:00100000:0.0:1789762690.146468:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2366:x1876702014854144:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762690.146493:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 256 00000100:00100000:2.0:1789762690.146510:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014854016:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762690.146573:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2366:x1876702014854144:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 109us (338us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762690.146582:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014854016:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 71us (272us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762690.146596:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 128 00000100:00100000:2.0:1789762690.146618:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 256 00000100:00100000:2.0:1789762690.167450:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022136832 00000100:00100000:2.0:1789762690.167458:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022136832 00000100:00100000:2.0:1789762690.167553:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57 00000100:00100000:2.0:1789762690.167569:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf567100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022136832:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762690.167608:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762690.167662:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf567100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022136832:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (274us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762690.167669:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57 00000100:00100000:2.0:1789762690.237628:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014854400 00000100:00100000:2.0:1789762690.237640:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014854400 00000100:00100000:2.0:1789762690.237696:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 213535691938 time 213535792757 state 0x0 ref 1 00000100:00100000:2.0:1789762690.237708:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 213735792757 time 213535792757 state 0x1 ref 1 00000100:00100000:0.0:1789762690.437668:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014854528 00000100:00100000:0.0:1789762690.437679:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014854528 00000100:00100000:0.0:1789762690.437727:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 213735792757 time 213735925948 state 0x0 ref 1 00000100:00100000:0.0:1789762690.437742:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 213935925948 time 213735925948 state 0x1 ref 1 00000100:00100000:2.0:1789762690.638052:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014854656 00000100:00100000:2.0:1789762690.638062:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014854656 00000100:00100000:2.0:1789762690.638127:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 213935925948 time 213936041682 state 0x0 ref 1 00000100:00100000:2.0:1789762690.638143:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 214136041682 time 213936041682 state 0x1 ref 1 00000100:00100000:2.0:1789762690.679274:0:3272:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789762690.679293:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc555c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022137088:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762690.679340:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022137344:0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.679378:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762690.679397:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762690.679457:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc1ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022137216:0@lo:400:kworker.0 00000100:00100000:0.0:1789762690.679483:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022137344 00000100:00100000:0.0:1789762690.679492:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022137344 00000100:00100000:2.0:1789762690.679492:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762690.679566:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022137216 00000100:00100000:0.0:1789762690.679571:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022137216 00000100:00100000:2.0:1789762690.679584:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:2.0:1789762690.679602:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022137344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762690.679614:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:0.0:1789762690.679625:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022137216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.679672:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022137344:12345-0@lo:400:kworker.0 Request processed in 78us (275us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762690.679689:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022137216:12345-0@lo:400:kworker.0 Request processed in 67us (198us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762690.679692:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00100000:0.0:1789762690.679708:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00100000:2.0:1789762690.679731:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789762690.679734:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762690.679741:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762690.679760:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022137472 00000100:00100000:0.0:1789762690.679765:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022137472 00000100:00100000:2.0:1789762690.679778:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc2d1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022136960:0@lo:400:kworker.0 00000100:00100000:0.0:1789762690.679810:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 132 00000100:00100000:0.0:1789762690.679826:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022137472:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.679862:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc1ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022137216:0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.679907:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc1d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022137472:0@lo:400:kworker.0 00000100:00100000:0.0:1789762690.679911:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022137472:12345-0@lo:400:kworker.0 Request processed in 90us (178us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762690.679919:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 132 00000100:00080000:0.0:1789762690.679940:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762690.679964:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc1c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022137344:0@lo:400:kworker.0 00000100:00100000:0.0:1789762690.680019:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc1e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022137600:0@lo:400:kworker.0 00000100:00100000:2.0:1789762690.838172:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762690.838242:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014854784 00000100:00100000:0.0:1789762690.838252:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014854784 00000100:00100000:0.0:1789762690.838296:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 214136041682 time 214136157160 state 0x0 ref 1 00000100:00100000:0.0:1789762690.838307:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 214336157160 time 214136157160 state 0x1 ref 1 00000100:00080000:0.0:1789762691.032450:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762691.037751:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762691.037806:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014854912 00000100:00100000:2.0:1789762691.037817:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014854912 00000100:00100000:2.0:1789762691.037867:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 214336157160 time 214336274742 state 0x0 ref 1 00000100:00100000:2.0:1789762691.037879:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 214536274742 time 214336274742 state 0x1 ref 1 00000100:00100000:0.0:1789762691.237799:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762691.237846:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014855040 00000100:00100000:2.0:1789762691.237857:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014855040 00000100:00100000:2.0:1789762691.237897:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 214536274742 time 214536367083 state 0x0 ref 1 00000100:00100000:2.0:1789762691.237915:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 214736367083 time 214536367083 state 0x1 ref 1 00000100:00100000:2.0:1789762691.438284:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762691.638082:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014855296 00000100:00100000:2.0:1789762691.638120:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014855296 00000100:00100000:2.0:1789762691.638184:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 214936475063 time 214936576828 state 0x0 ref 1 00000100:00100000:2.0:1789762691.638201:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 215136576828 time 214936576828 state 0x1 ref 1 00000100:00100000:2.0:1789762691.838443:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014855424 00000100:00100000:2.0:1789762691.838454:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014855424 00000100:00100000:2.0:1789762691.838500:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 215136576828 time 215136682667 state 0x0 ref 1 00000100:00100000:2.0:1789762691.838526:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 215336682667 time 215136682667 state 0x1 ref 1 00000100:00100000:2.0:1789762692.038462:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762692.038513:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014855552 00000100:00100000:0.0:1789762692.038524:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014855552 00000100:00100000:0.0:1789762692.038573:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 215336682667 time 215336779347 state 0x0 ref 1 00000100:00100000:0.0:1789762692.038584:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 215536779347 time 215336779347 state 0x1 ref 1 00000100:00100000:0.0:1789762692.238228:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762692.238284:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014855680 00000100:00100000:2.0:1789762692.238294:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014855680 00000100:00100000:2.0:1789762692.238336:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 215536779347 time 215536887647 state 0x0 ref 1 00000100:00100000:2.0:1789762692.238347:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 215736887647 time 215536887647 state 0x1 ref 1 00000100:00100000:0.0:1789762692.439413:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762692.439500:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014855808 00000100:00100000:2.0:1789762692.439511:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014855808 00000100:00100000:2.0:1789762692.439553:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 215736887647 time 215737025020 state 0x0 ref 1 00000100:00100000:2.0:1789762692.439565:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 215937025020 time 215737025020 state 0x1 ref 1 00000100:00100000:2.0:1789762692.638930:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762692.838905:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014856064 00000100:00100000:2.0:1789762692.838915:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014856064 00000100:00100000:2.0:1789762692.838959:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 216137154537 time 216137255032 state 0x0 ref 1 00000100:00100000:2.0:1789762692.838970:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 216337255032 time 216137255032 state 0x1 ref 1 00000100:00080000:0.0:1789762693.033591:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762693.038565:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762693.240405:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014856320 00000100:00100000:2.0:1789762693.240416:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014856320 00000100:00100000:2.0:1789762693.240469:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 216537374323 time 216537469068 state 0x0 ref 1 00000100:00100000:2.0:1789762693.240486:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 216737469068 time 216537469068 state 0x1 ref 1 00000100:00100000:2.0:1789762693.439265:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014856448 00000100:00100000:2.0:1789762693.439273:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014856448 00000100:00100000:2.0:1789762693.439306:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 216737469068 time 216737538985 state 0x0 ref 1 00000100:00100000:2.0:1789762693.439323:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 216937538985 time 216737538985 state 0x1 ref 1 00000100:00100000:2.0:1789762693.639151:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014856576 00000100:00100000:2.0:1789762693.639173:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014856576 00000100:00100000:2.0:1789762693.639205:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 216937538985 time 216937663237 state 0x0 ref 1 00000100:00100000:2.0:1789762693.639214:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217137663237 time 216937663237 state 0x1 ref 1 00010000:00010000:0.0:1789762693.837306:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b02562 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762693.837317:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b02562 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762693.837324:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b02562 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762693.837331:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812de00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762693.837334:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b02562 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762693.837340:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b02562 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762693.837356:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b02562 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762693.837367:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cceead42/0x416193e012b02562 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762693.837424:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfeded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014856576:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3527us (198340us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762693.837431:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 191 00000100:00100000:0.0:1789762693.837436:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217137663237 time 217137783137 state 0x2 ref 0 00000100:00100000:2.0:1789762693.839370:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014856704 00000100:00100000:2.0:1789762693.839378:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014856704 00000100:00100000:2.0:1789762693.839411:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217137663237 time 217137783137 state 0x0 ref 1 00000100:00100000:2.0:1789762693.839419:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217337783137 time 217137783137 state 0x1 ref 1 00000100:00100000:2.0:1789762694.040625:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014856832 00000100:00100000:2.0:1789762694.040637:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014856832 00000100:00100000:2.0:1789762694.040681:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217337783137 time 217337902732 state 0x0 ref 1 00000100:00100000:2.0:1789762694.040692:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217537902732 time 217337902732 state 0x1 ref 1 00010000:00010000:0.0:1789762694.071222:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762694.234178:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217537902732 time 217538080023 state 0x0 ref 1 00000100:00100000:0.0:1789762694.234208:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014856832:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762694.234248:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b02570 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.234263:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762694.234267:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762694.234271:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b02570 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.234283:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b02570 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.237449:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b02570 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.237460:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b02570 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.237483:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b02570 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.237489:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf54ce00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762694.237493:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b02570 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.237500:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b02570 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.237514:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b02570 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.237520:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b02570 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762694.237573:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014856832:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3380us (197003us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762694.237579:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 193 00000100:00100000:0.0:1789762694.237585:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217537902732 time 217538080023 state 0x2 ref 0 00000100:00100000:2.0:1789762694.239577:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014856960 00000100:00100000:2.0:1789762694.239591:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014856960 00000100:00100000:2.0:1789762694.239650:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217537902732 time 217538080023 state 0x0 ref 1 00000100:00100000:2.0:1789762694.239666:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217738080023 time 217538080023 state 0x1 ref 1 00000100:00100000:2.0:1789762694.439381:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762694.439464:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014857088 00000100:00100000:0.0:1789762694.439479:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014857088 00000100:00100000:0.0:1789762694.439530:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217738080023 time 217738182370 state 0x0 ref 1 00000100:00100000:0.0:1789762694.439541:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217938182370 time 217738182370 state 0x1 ref 1 00000100:00100000:0.0:1789762694.634380:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217938182370 time 217938282293 state 0x0 ref 1 00000100:00100000:0.0:1789762694.634410:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014857088:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762694.634446:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0257e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.634461:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762694.634465:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762694.634469:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0257e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.634480:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0257e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.637651:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0257e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.637663:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0257e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.637673:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0257e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.637682:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf54ce00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762694.637688:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0257e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.637717:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0257e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.637735:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0257e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762694.637744:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0257e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762694.637809:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014857088:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3412us (198428us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762694.637819:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 195 00000100:00100000:0.0:1789762694.637825:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 217938182370 time 217938282293 state 0x2 ref 0 00000100:00100000:2.0:1789762694.639684:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762694.839940:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014857344 00000100:00100000:0.0:1789762694.839953:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014857344 00000100:00100000:0.0:1789762694.840002:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 218138282293 time 218138346347 state 0x0 ref 1 00000100:00100000:0.0:1789762694.840014:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 218338346347 time 218138346347 state 0x1 ref 1 00000100:00100000:0.0:1789762695.034521:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 218338346347 time 218338421098 state 0x0 ref 1 00000100:00100000:0.0:1789762695.034579:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014857344:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:2.0:1789762695.034632:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762695.034646:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0258c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762695.034695:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762695.034699:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762695.034706:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0258c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762695.034721:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009028d60f/0x416193e012b0258c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762695.240035:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762695.240128:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014857600 00000100:00100000:0.0:1789762695.240142:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014857600 00000100:00100000:0.0:1789762695.240205:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 218538421098 time 218538485808 state 0x0 ref 1 00000100:00100000:0.0:1789762695.240218:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 218738485808 time 218538485808 state 0x1 ref 1 00000100:00100000:2.0:1789762695.266215:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014857728 00000100:00100000:2.0:1789762695.266236:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014857728 00000100:00100000:0.0:1789762695.266268:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014857984 00000100:00100000:0.0:1789762695.266284:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014857984 00000100:00100000:2.0:1789762695.266311:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 258 00000100:00100000:2.0:1789762695.266326:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014857728:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762695.266365:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 134 00000100:00100000:0.0:1789762695.266384:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014857984:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762695.266395:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014857728:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 72us (248us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762695.266404:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 258 00000100:00100000:0.0:1789762695.266468:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014857984:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 88us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762695.266481:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 134 00000100:00100000:2.0:1789762695.287403:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022137728 00000100:00100000:2.0:1789762695.287410:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022137728 00000100:00100000:2.0:1789762695.287490:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 59 00000100:00100000:2.0:1789762695.287502:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022137728:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762695.287531:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762695.287600:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022137728:12345-0@lo:13:osp-pre-0-0.0 Request processed in 98us (245us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762695.287608:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 59 00000100:00100000:2.0:1789762695.640435:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762695.640478:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014858240 00000100:00100000:0.0:1789762695.640487:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014858240 00000100:00100000:0.0:1789762695.640519:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 218938591438 time 218938669632 state 0x0 ref 1 00000100:00100000:0.0:1789762695.640528:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 219138669632 time 218938669632 state 0x1 ref 1 00000100:00100000:2.0:1789762695.799306:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad00380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022137984:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762695.799380:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762695.799435:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc1c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022138112:0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.799490:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc1f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022138496:0@lo:400:kworker.0 00000100:00100000:0.0:1789762695.799499:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762695.799545:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc1ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022138624:0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.799553:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762695.799621:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762695.799652:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022138368 00000100:00100000:2.0:1789762695.799658:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022138368 00000100:00100000:0.0:1789762695.799665:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789762695.799674:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc1fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022138240:0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.799716:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022138624 00000100:00100000:2.0:1789762695.799720:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022138624 00000100:00100000:0.0:1789762695.799735:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762695.799761:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 67 00000100:00100000:2.0:1789762695.799777:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022138368:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762695.799808:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022138112 00000100:00100000:0.0:1789762695.799825:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022138112 00000100:00100000:2.0:1789762695.799839:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022138368:12345-0@lo:400:kworker.0 Request processed in 66us (275us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762695.799864:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 67 00000100:00100000:2.0:1789762695.799898:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 68 00000100:00100000:0.0:1789762695.799901:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022138496 00000100:00100000:0.0:1789762695.799909:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022138496 00000100:00100000:2.0:1789762695.799912:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022138624:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.799975:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022138624:12345-0@lo:400:kworker.0 Request processed in 69us (360us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762695.799984:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 68 00000100:00100000:0.0:1789762695.800003:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 135 00000100:00100000:0.0:1789762695.800020:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022138112:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762695.800053:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc1ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022138624:0@lo:400:kworker.0 00000100:00100000:0.0:1789762695.800138:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022138112:12345-0@lo:400:kworker.0 Request processed in 77us (597us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762695.800157:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc1c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022138112:0@lo:400:kworker.0 00000100:00100000:0.0:1789762695.800173:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1789762695.800201:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc1f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022138496:0@lo:400:kworker.0 00000100:00100000:0.0:1789762695.800208:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 137 00000100:00100000:0.0:1789762695.800222:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022138240:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762695.800308:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022138240:12345-0@lo:400:kworker.0 Request processed in 88us (578us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762695.800323:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad00380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022137984:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762695.800324:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000100:00100000:2.0:1789762695.840582:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762696.040885:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762696.040935:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014858496 00000100:00100000:0.0:1789762696.040945:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014858496 00000100:00100000:0.0:1789762696.040979:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 219338774953 time 219338847532 state 0x0 ref 1 00000100:00100000:0.0:1789762696.040988:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 219538847532 time 219338847532 state 0x1 ref 1 00000100:00100000:0.0:1789762696.239969:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762696.440608:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762696.440690:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014858752 00000100:00100000:2.0:1789762696.440699:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014858752 00000100:00100000:2.0:1789762696.440728:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 219738948297 time 219739035548 state 0x0 ref 1 00000100:00100000:2.0:1789762696.440737:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 219939035548 time 219739035548 state 0x1 ref 1 00000100:00100000:2.0:1789762696.640077:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762696.841713:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014859008 00000100:00100000:2.0:1789762696.841721:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014859008 00000100:00100000:2.0:1789762696.841751:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 220139125998 time 220139190583 state 0x0 ref 1 00000100:00100000:2.0:1789762696.841766:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 220339190583 time 220139190583 state 0x1 ref 1 00000100:00080000:0.0:1789762697.035451:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762697.241447:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014859264 00000100:00100000:2.0:1789762697.241462:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014859264 00000100:00100000:2.0:1789762697.241493:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 220539261290 time 220539319883 state 0x0 ref 1 00000100:00100000:2.0:1789762697.241502:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 220739319883 time 220539319883 state 0x1 ref 1 00010000:00010000:0.0:1789762697.440073:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001df8296d/0x416193e012b025e0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.440118:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001df8296d/0x416193e012b025e0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.440150:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001df8296d/0x416193e012b025e0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.440159:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762697.440165:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001df8296d/0x416193e012b025e0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.440174:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001df8296d/0x416193e012b025e0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.440194:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001df8296d/0x416193e012b025e0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.440201:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001df8296d/0x416193e012b025e0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762697.440269:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014859264:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4707us (198862us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762697.440299:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 209 00000100:00100000:0.0:1789762697.440309:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 220739319883 time 220739432324 state 0x2 ref 0 00000100:00100000:0.0:1789762697.442290:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762697.641458:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014859520 00000100:00100000:2.0:1789762697.641466:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014859520 00000100:00100000:2.0:1789762697.641505:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 220939432324 time 220939500728 state 0x0 ref 1 00000100:00100000:2.0:1789762697.641517:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 221139500728 time 220939500728 state 0x1 ref 1 00000100:00100000:0.0:1789762697.835689:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 221139500728 time 221139591980 state 0x0 ref 1 00000100:00100000:0.0:1789762697.835717:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014859520:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762697.835754:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025ee lrc: 3/1,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.835768:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762697.835771:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762697.835776:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025ee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.835788:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025ee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.838928:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025ee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.838939:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025ee lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.838946:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025ee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.838969:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762697.838972:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025ee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.838980:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025ee lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.838993:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025ee lrc: 2/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762697.838998:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025ee lrc: 0/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762697.839049:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014859520:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3343us (197633us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762697.839060:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 211 00000100:00100000:0.0:1789762697.839067:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 221139500728 time 221139591980 state 0x2 ref 0 00000100:00100000:2.0:1789762697.840984:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762698.041197:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762698.041249:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014859776 00000100:00100000:0.0:1789762698.041257:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014859776 00000100:00100000:0.0:1789762698.041290:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 221339591980 time 221339659377 state 0x0 ref 1 00000100:00100000:0.0:1789762698.041299:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 221539659377 time 221339659377 state 0x1 ref 1 00010000:00010000:0.0:1789762698.167207:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762698.235822:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 221539659377 time 221539724952 state 0x0 ref 1 00000100:00100000:0.0:1789762698.235862:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014859776:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762698.235899:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025fc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.235912:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762698.235916:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762698.235920:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.235931:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.238742:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.238758:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025fc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.238769:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.238778:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762698.238783:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.238794:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025fc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.238822:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025fc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.238828:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b025fc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762698.238877:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014859776:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3022us (197680us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762698.238883:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 213 00000100:00100000:0.0:1789762698.238888:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 221539659377 time 221539724952 state 0x2 ref 0 00000100:00100000:0.0:1789762698.441517:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014860032 00000100:00100000:0.0:1789762698.441526:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014860032 00000100:00100000:0.0:1789762698.441558:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 221739724952 time 221739777672 state 0x0 ref 1 00000100:00100000:0.0:1789762698.441567:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 221939777672 time 221739777672 state 0x1 ref 1 00000100:00100000:0.0:1789762698.635934:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 221939777672 time 221939837083 state 0x0 ref 1 00000100:00100000:0.0:1789762698.635962:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014860032:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762698.635998:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0260a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.636012:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762698.636016:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762698.636021:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0260a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.636047:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0260a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.639352:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0260a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.639362:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0260a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.639370:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0260a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.639376:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762698.639379:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0260a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.639386:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0260a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.639400:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0260a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762698.639406:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0260a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762698.639460:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014860032:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3511us (198005us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762698.639468:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 215 00000100:00100000:0.0:1789762698.639473:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 221939777672 time 221939837083 state 0x2 ref 0 00000100:00100000:2.0:1789762698.841733:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762698.841790:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014860288 00000100:00100000:0.0:1789762698.841798:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014860288 00000100:00100000:0.0:1789762698.841831:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 222139837083 time 222139910990 state 0x0 ref 1 00000100:00100000:0.0:1789762698.841840:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 222339910990 time 222139910990 state 0x1 ref 1 00000100:00100000:0.0:1789762699.036078:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 222339910990 time 222339980150 state 0x0 ref 1 00000100:00100000:0.0:1789762699.036164:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014860288:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762699.036210:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02618 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.036237:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762699.036242:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762699.036250:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02618 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.036266:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02618 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.039238:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02618 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.039257:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02618 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.039265:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02618 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.039270:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762699.039274:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02618 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.039281:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02618 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.039294:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02618 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.039299:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02618 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762699.039346:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014860288:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3229us (197613us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762699.039354:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 217 00000100:00100000:0.0:1789762699.039359:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 222339910990 time 222339980150 state 0x2 ref 0 00000100:00100000:2.0:1789762699.041467:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762699.241340:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014860544 00000100:00100000:0.0:1789762699.241354:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014860544 00000100:00100000:0.0:1789762699.241385:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 222539980150 time 222540058135 state 0x0 ref 1 00000100:00100000:0.0:1789762699.241394:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 222740058135 time 222540058135 state 0x1 ref 1 00000100:00100000:0.0:1789762699.436215:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 222740058135 time 222740118025 state 0x0 ref 1 00000100:00100000:0.0:1789762699.436239:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014860544:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762699.436274:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02626 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.436288:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762699.436291:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762699.436296:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02626 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.436306:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02626 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.439674:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02626 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.439686:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02626 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.439694:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02626 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.439712:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762699.439717:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02626 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.439724:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02626 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.439740:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02626 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.439749:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02626 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762699.439809:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014860544:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3578us (198520us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762699.439822:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 219 00000100:00100000:0.0:1789762699.439829:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 222740058135 time 222740118025 state 0x2 ref 0 00000100:00100000:0.0:1789762699.442040:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762699.641650:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762699.641706:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014860800 00000100:00100000:2.0:1789762699.641716:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014860800 00000100:00100000:2.0:1789762699.641756:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 222940118025 time 222940184813 state 0x0 ref 1 00000100:00100000:2.0:1789762699.641768:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 223140184813 time 222940184813 state 0x1 ref 1 00000100:00100000:0.0:1789762699.836364:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 223140184813 time 223140266158 state 0x0 ref 1 00000100:00100000:0.0:1789762699.836392:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014860800:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762699.836429:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02634 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.836458:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762699.836461:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762699.836466:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02634 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762699.836479:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02634 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x8:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762700.041389:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762700.041443:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014861056 00000100:00100000:0.0:1789762700.041452:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014861056 00000100:00100000:0.0:1789762700.041484:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 223340266158 time 223340332267 state 0x0 ref 1 00000100:00100000:0.0:1789762700.041493:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 223540332267 time 223340332267 state 0x1 ref 1 00000100:00100000:2.0:1789762700.386413:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762700.386564:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762700.386628:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:0.0:1789762700.386661:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762700.386662:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014861568 00000100:00100000:2.0:1789762700.386668:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014861568 00000100:00100000:2.0:1789762700.386721:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 139 00000100:00100000:2.0:1789762700.386729:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014861568:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762700.386764:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014861568:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 35us (139us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762700.386784:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 139 00000100:00100000:2.0:1789762700.407403:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022138752 00000100:00100000:2.0:1789762700.407412:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022138752 00000100:00100000:2.0:1789762700.407473:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 61 00000100:00100000:2.0:1789762700.407484:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022138752:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762700.407524:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762700.407580:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022138752:12345-0@lo:13:osp-pre-0-0.0 Request processed in 98us (228us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762700.407586:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 61 00000100:00100000:0.0:1789762700.407628:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad02300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022138752:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762700.442053:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762700.442119:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014861696 00000100:00100000:0.0:1789762700.442128:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014861696 00000100:00100000:0.0:1789762700.442161:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 223740422327 time 223740470713 state 0x0 ref 1 00000100:00100000:0.0:1789762700.442169:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 223940470713 time 223740470713 state 0x1 ref 1 00000100:00100000:2.0:1789762700.642149:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762700.842801:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762700.842850:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014861952 00000100:00100000:2.0:1789762700.842859:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014861952 00000100:00100000:2.0:1789762700.842891:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 224140586028 time 224140662467 state 0x0 ref 1 00000100:00100000:2.0:1789762700.842901:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 224340662467 time 224140662467 state 0x1 ref 1 00000100:00100000:0.0:1789762700.919265:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022138880:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762700.919275:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb8d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022139136:0@lo:400:kworker.0 00000100:00100000:0.0:1789762700.919341:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762700.919342:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762700.919420:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb8f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022139648:0@lo:400:kworker.0 00000100:00100000:0.0:1789762700.919429:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb8fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022139392:0@lo:400:kworker.0 00000100:00100000:2.0:1789762700.919469:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762700.919523:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762700.919536:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022139264:0@lo:400:kworker.0 00000100:00100000:2.0:1789762700.919579:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762700.919625:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022139008 00000100:00100000:2.0:1789762700.919630:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022139008 00000100:00100000:2.0:1789762700.919685:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:2.0:1789762700.919708:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022139008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762700.919764:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022139008:12345-0@lo:400:kworker.0 Request processed in 62us (426us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762700.919771:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00100000:2.0:1789762700.919811:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:2.0:1789762700.919824:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022139264:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762700.919877:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022139264:12345-0@lo:400:kworker.0 Request processed in 57us (299us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762700.919889:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:2.0:1789762700.919922:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022139392 00000100:00100000:2.0:1789762700.919926:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022139392 00000100:00100000:2.0:1789762700.919962:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 70 00000100:00100000:2.0:1789762700.919971:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022139392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762700.920021:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022139392:12345-0@lo:400:kworker.0 Request processed in 50us (139us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762700.920028:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 70 00000100:00100000:2.0:1789762700.920067:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb8fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022139392:0@lo:400:kworker.0 00000100:00100000:0.0:1789762701.042194:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762701.242577:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762701.242626:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014862208 00000100:00100000:2.0:1789762701.242634:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014862208 00000100:00100000:2.0:1789762701.242674:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 224540757283 time 224540835070 state 0x0 ref 1 00000100:00100000:2.0:1789762701.242686:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 224740835070 time 224540835070 state 0x1 ref 1 00000100:00100000:2.0:1789762701.643519:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014862464 00000100:00100000:2.0:1789762701.643527:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014862464 00000100:00100000:2.0:1789762701.643561:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 224940944358 time 224941032070 state 0x0 ref 1 00000100:00100000:2.0:1789762701.643570:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 225141032070 time 224941032070 state 0x1 ref 1 00000100:00100000:0.0:1789762701.842458:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762702.042542:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762702.042603:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014862720 00000100:00100000:0.0:1789762702.042611:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014862720 00000100:00100000:0.0:1789762702.042650:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 225341115843 time 225341165070 state 0x0 ref 1 00000100:00100000:0.0:1789762702.042663:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 225541165070 time 225341165070 state 0x1 ref 1 00010000:00010000:0.0:1789762702.263207:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762702.442671:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762702.442724:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014862976 00000100:00100000:2.0:1789762702.442734:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014862976 00000100:00100000:2.0:1789762702.442772:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 225741257467 time 225741352563 state 0x0 ref 1 00000100:00100000:2.0:1789762702.442781:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key gid={500} rule@0xffff981eb5967a00 gen 0 rate 5 depth 3 token 0 deadline 225941352563 time 225741352563 state 0x1 ref 1 00000100:00100000:0.0:1789762702.642545:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762702.675489:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762702.679495:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762702.681240:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014863872 00000100:00100000:0.0:1789762702.681246:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014863872 00000100:00100000:0.0:1789762702.681275:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 38 00000100:00100000:0.0:1789762702.681282:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:11298:x1876702014863872:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:0.0:1789762702.683055:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:11298:x1876702014863872:12345-192.168.203.48@tcp:37:rm.0 Request processed in 1772us (1861us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762702.683062:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 38 00000100:00100000:2.0:1789762702.684695:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014864128 00000100:00100000:2.0:1789762702.684701:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014864128 00000100:00100000:2.0:1789762702.684733:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 39 00000100:00100000:2.0:1789762702.684740:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:11298:x1876702014864128:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:2.0:1789762702.684851:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:11298:x1876702014864128:12345-192.168.203.48@tcp:35:rm.0 Request processed in 110us (233us total) trans 4294967408 rc 0/0 00000100:00100000:2.0:1789762702.684860:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 39 00000100:00100000:2.0:1789762702.685805:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014864384 00000100:00100000:2.0:1789762702.685811:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014864384 00000100:00100000:2.0:1789762702.685844:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 143 00000100:00100000:2.0:1789762702.685853:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:11298:x1876702014864384:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:2.0:1789762702.685864:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762702.685875:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026b9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd23b expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.685911:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdec00/0x416193e012b026c0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.685919:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdec00/0x416193e012b026c0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.685935:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdec00/0x416193e012b026c0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762702.685942:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 16384 00010000:00010000:2.0:1789762702.685986:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026b9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd23b expref: 16 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.685997:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdec00/0x416193e012b026c0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x13:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd23b expref: 15 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.686002:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26000) 00010000:00010000:2.0:1789762702.686004:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1789762702.686006:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87fdec00, rc 0) 00000100:00100000:2.0:1789762702.686032:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:11298:x1876702014864384:12345-192.168.203.48@tcp:101:rm.0 Request processed in 182us (278us total) trans 4294967409 rc 0/0 00000100:00100000:2.0:1789762702.686037:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 143 00000100:00100000:0.0:1789762702.686954:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014864512 00000100:00100000:0.0:1789762702.686959:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014864512 00000100:00100000:0.0:1789762702.686985:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 144 00000100:00100000:0.0:1789762702.686991:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:11298:x1876702014864512:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:0.0:1789762702.687027:0:5806:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7610564608 00000100:00100000:0.0:1789762702.687063:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:11298:x1876702014864512:12345-192.168.203.48@tcp:41:rm.0 Request processed in 72us (156us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762702.687069:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 144 00000100:00100000:0.0:1789762702.688011:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014864768 00000100:00100000:0.0:1789762702.688022:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014864768 00000100:00100000:0.0:1789762702.688050:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 145 00000100:00100000:0.0:1789762702.688059:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfeced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:11298:x1876702014864768:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:0.0:1789762702.688127:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1789762702.688144:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b023cc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd21f expref: 15 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.688164:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31600/0x416193e012b0207d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1fc expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.688176:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x15:0x0].0x0 (ffff981e82a26400) 00010000:00010000:0.0:1789762702.688189:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ca00/0x416193e012b026a4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd22d expref: 13 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.688200:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26000) 00010000:00010000:0.0:1789762702.688204:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762702.688216:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd800/0x416193e012b026ce lrc: 3/0,1 mode: --/CW res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.688226:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd800/0x416193e012b026ce lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.688234:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd800/0x416193e012b026ce lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.688247:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b026d5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.688256:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b026d5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.688275:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b026d5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.688549:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bca00/0x416193e012b026dc lrc: 3/0,1 mode: --/EX res: [0x200000401:0x15:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.688559:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bca00/0x416193e012b026dc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x15:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.688568:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bca00/0x416193e012b026dc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x15:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762702.689393:0:6611:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].8, 64 off8704 00000004:00080000:0.0:1789762702.689404:0:6611:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 225993316088ns, next at 226993313902ns, rc = 0 00010000:00010000:0.0:1789762702.689451:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b026e3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689461:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b026e3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689471:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b026e3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689479:0:6611:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b026e3 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689487:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b026e3 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689494:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b026e3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x15:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689525:0:6611:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762702.689540:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bca00/0x416193e012b026dc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x15:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689549:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bca00/0x416193e012b026dc lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x15:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689561:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bca00/0x416193e012b026dc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x15:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689582:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bca00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762702.689592:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bca00/0x416193e012b026dc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x15:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689615:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bca00/0x416193e012b026dc lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x15:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689649:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x15:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762702.689653:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762702.689658:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bca00/0x416193e012b026dc lrc: 2/0,0 mode: --/EX res: [0x200000401:0x15:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689675:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bca00/0x416193e012b026dc lrc: 0/0,0 mode: --/EX res: [0x200000401:0x15:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689710:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd800/0x416193e012b026ce lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689719:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd800/0x416193e012b026ce lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689738:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd800/0x416193e012b026ce lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689744:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bd800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762702.689748:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd800/0x416193e012b026ce lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689756:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd800/0x416193e012b026ce lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689766:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26000) 00010000:00010000:0.0:1789762702.689769:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762702.689772:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd800/0x416193e012b026ce lrc: 2/0,0 mode: --/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.689779:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd800/0x416193e012b026ce lrc: 0/0,0 mode: --/CW res: [0x200000401:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762702.689790:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007f8d1b80 reply state 00000000386d22b3 transno 4294967410 ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b026d5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762702.689799:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762702.689842:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfeced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:11298:x1876702014864768:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1783us (1871us total) trans 4294967410 rc 0/0 00000100:00100000:0.0:1789762702.689850:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 145 00010000:00010000:0.0:1789762702.690368:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b026d5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.690378:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b026d5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.690397:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b026d5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.690404:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bd200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762702.690407:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b026d5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.690414:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b026d5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.690424:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x4009f790 (ffff981ea0aa3c00) 00010000:00010000:0.0:1789762702.690427:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762702.690430:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b026d5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762702.690437:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b026d5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x13:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762702.691071:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014865024 00000100:00100000:0.0:1789762702.691075:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014865024 00000100:00100000:0.0:1789762702.691136:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 40 00000100:00100000:0.0:1789762702.691144:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b19c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:11298:x1876702014865024:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:0.0:1789762702.691225:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b19c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:11298:x1876702014865024:12345-192.168.203.48@tcp:35:rm.0 Request processed in 82us (187us total) trans 4294967411 rc 0/0 00000100:00100000:0.0:1789762702.691230:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 40 00000100:00100000:2.0:1789762702.692077:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014865280 00000100:00100000:2.0:1789762702.692083:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014865280 00000100:00100000:2.0:1789762702.692153:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 146 00000100:00100000:2.0:1789762702.692161:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:11298:x1876702014865280:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:2.0:1789762702.692185:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762702.692198:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdec00/0x416193e012b026c0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x13:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd23b expref: 14 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692216:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac30c00/0x416193e012b0204c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1ee expref: 13 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692241:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26000) 00010000:00010000:2.0:1789762702.692257:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebac31000/0x416193e012b01fea lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd1d9 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692267:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762702.692273:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762702.692283:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026ea lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692290:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026ea lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692295:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026ea lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692303:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b026f1 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692307:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b026f1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692312:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b026f1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692515:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b026f8 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692522:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b026f8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692535:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b026f8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762702.692880:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x13:0x0]: returning size 16384 00010000:00010000:2.0:1789762702.692891:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b026f8 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692899:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b026f8 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692906:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b026f8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692912:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fde400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762702.692916:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b026f8 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692923:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b026f8 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x13:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692934:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762702.692937:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762702.692940:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b026f8 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692947:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b026f8 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x13:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692965:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026ea lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692972:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026ea lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692986:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026ea lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.692992:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fdfa00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762702.692996:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026ea lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.693004:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026ea lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.693014:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26500) 00010000:00010000:2.0:1789762702.693016:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762702.693020:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026ea lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.693027:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b026ea lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762702.693040:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000046305292 reply state 0000000093da5ed3 transno 4294967412 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b026f1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762702.693049:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762702.693123:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:11298:x1876702014865280:12345-192.168.203.48@tcp:36:rm.0 Request processed in 961us (1100us total) trans 4294967412 rc 0/0 00000100:00100000:2.0:1789762702.693132:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 146 00010000:00010000:2.0:1789762702.693743:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b026f1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.693754:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b026f1 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.693773:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b026f1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.693779:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fdf600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762702.693783:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b026f1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.693790:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b026f1 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.693800:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981e82a26900) 00010000:00010000:2.0:1789762702.693804:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762702.693807:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b026f1 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762702.693813:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b026f1 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762703.539663:0:12761:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 226843573042ns, next at 226993313902ns, rc = 0 00000100:00080000:0.0:1789762703.548231:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:0.0:1789762703.548993:0:12761:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 226852905140ns, next at 194503143298ns, rc = 0 00010000:00010000:0.0:1789762704.311215:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762705.506431:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014865536 00000100:00100000:0.0:1789762705.506447:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014865536 00000100:00100000:2.0:1789762705.506494:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014865408 00000100:00100000:2.0:1789762705.506502:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014865408 00000100:00100000:0.0:1789762705.506551:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 143 00000100:00100000:2.0:1789762705.506559:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 262 00000100:00100000:0.0:1789762705.506570:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfece680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014865536:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762705.506579:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850de050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014865408:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762705.506667:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfece680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014865536:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 101us (343us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762705.506670:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850de050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014865408:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 98us (267us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762705.506701:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 143 00000100:00100000:2.0:1789762705.506708:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 262 00000100:00080000:0.0:1789762705.506742:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762705.506748:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762705.527295:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022139904:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762705.527373:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762705.527445:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022139904 00000100:00100000:2.0:1789762705.527451:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022139904 00000100:00080000:0.0:1789762705.527503:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762705.527516:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 63 00000100:00100000:2.0:1789762705.527525:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022139904:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762705.527553:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762705.527590:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022139904:12345-0@lo:13:osp-pre-0-0.0 Request processed in 68us (217us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762705.527595:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 63 00000100:00100000:0.0:1789762705.527654:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022139904:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762706.039323:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022140160:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762706.039403:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762706.039478:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022140032 00000100:00100000:2.0:1789762706.039490:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022140032 00000100:00100000:0.0:1789762706.039507:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022140544:0@lo:400:kworker.0 00000100:00100000:2.0:1789762706.039569:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 263 00000100:00100000:2.0:1789762706.039583:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022140032:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762706.039593:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022140416:0@lo:400:kworker.0 00000100:00100000:0.0:1789762706.039659:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762706.039683:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022140032:12345-0@lo:400:kworker.0 Request processed in 104us (286us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762706.039700:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 263 00000100:00100000:0.0:1789762706.039770:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022140288 00000100:00100000:0.0:1789762706.039786:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022140288 00000100:00100000:2.0:1789762706.039795:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022140672 00000100:00100000:2.0:1789762706.039805:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022140672 00000100:00100000:0.0:1789762706.039863:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022140416 00000100:00100000:0.0:1789762706.039869:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022140416 00000100:00100000:2.0:1789762706.039871:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 145 00000100:00100000:2.0:1789762706.039895:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022140288:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762706.039927:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:0.0:1789762706.039945:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022140672:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762706.039984:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022140288:12345-0@lo:400:kworker.0 Request processed in 93us (563us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762706.040028:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00000100:00100000:0.0:1789762706.040035:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022140672:12345-0@lo:400:kworker.0 Request processed in 89us (432us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762706.040054:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00080000:0.0:1789762706.040165:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762706.040185:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762706.040207:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022140288:0@lo:400:kworker.0 00000100:00100000:0.0:1789762706.040232:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfece300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022140672:0@lo:400:kworker.0 00000100:00100000:2.0:1789762706.040262:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022140800:0@lo:400:kworker.0 00010000:00010000:0.0:1789762708.407205:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762710.626269:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762710.626320:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762710.626326:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014866048 00000100:00100000:0.0:1789762710.626340:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014866048 00000100:00100000:2.0:1789762710.626346:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762710.626376:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762710.626406:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014866304 00000100:00100000:2.0:1789762710.626413:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014866304 00000100:00100000:0.0:1789762710.626437:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 148 00000100:00100000:0.0:1789762710.626452:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014866048:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762710.626471:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 149 00000100:00100000:2.0:1789762710.626485:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014866304:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762710.626548:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014866048:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 96us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762710.626550:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014866304:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 65us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762710.626568:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 148 00000100:00100000:2.0:1789762710.626575:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 149 00000100:00080000:0.0:1789762710.626594:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762710.626613:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014866176 00000100:00100000:2.0:1789762710.626619:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014866176 00000100:00080000:0.0:1789762710.626652:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762710.626677:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 74 00000100:00100000:2.0:1789762710.626687:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014866176:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762710.626732:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014866176:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 47us (355us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762710.626737:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 74 00000100:00100000:2.0:1789762710.648392:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022140928 00000100:00100000:2.0:1789762710.648402:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022140928 00000100:00100000:2.0:1789762710.648481:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 65 00000100:00100000:2.0:1789762710.648495:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022140928:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762710.648537:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762710.648599:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022140928:12345-0@lo:13:osp-pre-0-0.0 Request processed in 106us (248us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762710.648610:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 65 00000100:00100000:2.0:1789762711.159323:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762711.159357:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022141056:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762711.159364:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022141184:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.159442:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762711.159506:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762711.159565:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022141056 00000100:00100000:2.0:1789762711.159575:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022141056 00000100:00100000:0.0:1789762711.159602:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022141696:0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.159637:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 66 00000100:00100000:2.0:1789762711.159647:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022141056:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762711.159691:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762711.159694:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022141184 00000100:00100000:0.0:1789762711.159704:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022141184 00000100:00100000:2.0:1789762711.159758:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022141056:12345-0@lo:13:osp-pre-1-0.0 Request processed in 111us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762711.159768:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 66 00000100:00100000:0.0:1789762711.159777:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 265 00000100:00100000:0.0:1789762711.159795:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850df450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022141184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.159847:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022141056:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762711.159873:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850df450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022141184:12345-0@lo:400:kworker.0 Request processed in 82us (378us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762711.159896:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 265 00000100:00100000:2.0:1789762711.159901:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022141312 00000100:00100000:2.0:1789762711.159906:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022141312 00000100:00100000:2.0:1789762711.159944:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022141440 00000100:00100000:2.0:1789762711.159948:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022141440 00000100:00100000:0.0:1789762711.159973:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:0.0:1789762711.159999:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfece300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022141312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.159999:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:2.0:1789762711.160015:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022141440:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.160163:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022141440:12345-0@lo:400:kworker.0 Request processed in 146us (481us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762711.160168:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfece300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022141312:12345-0@lo:400:kworker.0 Request processed in 178us (669us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762711.160182:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00100000:0.0:1789762711.160199:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:2.0:1789762711.160209:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022141696 00000100:00100000:2.0:1789762711.160213:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022141696 00000100:00100000:2.0:1789762711.160274:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1789762711.160291:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022141696:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762711.160339:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022141696:12345-0@lo:400:kworker.0 Request processed in 54us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762711.160348:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00010000:00010000:0.0:1789762712.503217:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762714.551228:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762715.746303:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762715.746349:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762715.746369:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014866560 00000100:00100000:2.0:1789762715.746375:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762715.746380:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014866560 00000100:00100000:2.0:1789762715.746404:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762715.746435:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014866688 00000100:00100000:2.0:1789762715.746442:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014866688 00000100:00100000:0.0:1789762715.746448:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 153 00000100:00100000:0.0:1789762715.746463:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014866560:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762715.746492:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 77 00000100:00100000:2.0:1789762715.746512:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014866688:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762715.746533:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014866560:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 74us (236us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762715.746540:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 153 00000100:00100000:2.0:1789762715.746575:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014866688:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 66us (166us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762715.746591:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 77 00000100:00100000:2.0:1789762715.767224:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022141952:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762715.767266:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762715.767309:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022141952 00000100:00100000:0.0:1789762715.767315:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022141952 00000100:00100000:0.0:1789762715.767365:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 67 00000100:00100000:0.0:1789762715.767374:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022141952:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762715.767397:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762715.767445:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022141952:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (179us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762715.767451:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 67 00000100:00100000:2.0:1789762715.767485:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022141952:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762716.279239:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022142464:0@lo:400:kworker.0 00000100:00100000:0.0:1789762716.279251:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022142080:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762716.279316:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762716.279318:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762716.279382:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022142720:0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.279406:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022142464 00000100:00100000:0.0:1789762716.279413:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762716.279414:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022142464 00000100:00100000:2.0:1789762716.279474:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022142336 00000100:00100000:0.0:1789762716.279480:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022142208 00000100:00100000:2.0:1789762716.279480:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022142336 00000100:00100000:0.0:1789762716.279487:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022142208 00000100:00100000:2.0:1789762716.279515:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:2.0:1789762716.279527:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022142464:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762716.279529:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:0.0:1789762716.279545:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022142208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.279585:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022142464:12345-0@lo:400:kworker.0 Request processed in 63us (272us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762716.279592:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:2.0:1789762716.279623:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:0.0:1789762716.279628:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022142208:12345-0@lo:400:kworker.0 Request processed in 86us (306us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762716.279645:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:2.0:1789762716.279652:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022142592:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762716.279692:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762716.279697:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762716.279708:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022142592:12345-0@lo:400:kworker.0 Request processed in 63us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762716.279716:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:0.0:1789762716.279736:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022142208:0@lo:400:kworker.0 00000100:00100000:0.0:1789762716.279758:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022142592:0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.279766:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022142464:0@lo:400:kworker.0 00000100:00100000:2.0:1789762716.279843:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022142080:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762716.279863:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022142720:0@lo:400:kworker.0 00010000:00010000:0.0:1789762716.599274:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762718.647194:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762720.695239:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762720.866796:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762720.866831:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762720.866864:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762720.866890:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762720.866944:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014867072 00000100:00100000:2.0:1789762720.866966:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014867072 00000100:00100000:0.0:1789762720.866967:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014867328 00000100:00100000:0.0:1789762720.866973:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014867328 00000100:00100000:2.0:1789762720.867037:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 158 00000100:00100000:0.0:1789762720.867049:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 159 00000100:00100000:2.0:1789762720.867056:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014867072:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762720.867066:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014867328:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762720.867155:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014867072:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 105us (318us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762720.867167:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014867328:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 103us (273us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762720.867181:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 158 00000100:00100000:0.0:1789762720.867195:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 159 00000100:00080000:0.0:1789762720.867232:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762720.867239:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762720.887301:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789762720.887324:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022142848:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762720.887400:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762720.887522:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022142848 00000100:00100000:2.0:1789762720.887529:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022142848 00000100:00100000:2.0:1789762720.887608:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 69 00000100:00100000:2.0:1789762720.887619:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022142848:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762720.887666:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762720.887732:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022142848:12345-0@lo:13:osp-pre-0-0.0 Request processed in 113us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762720.887742:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 69 00000100:00100000:0.0:1789762721.399400:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022142976:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762721.399402:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb8ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022143232:0@lo:400:kworker.0 00000100:00100000:0.0:1789762721.399533:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762721.399617:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022142976 00000100:00100000:2.0:1789762721.399645:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022142976 00000100:00100000:0.0:1789762721.399676:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb8c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022143488:0@lo:400:kworker.0 00000100:00100000:0.0:1789762721.399761:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022143360 00000100:00100000:0.0:1789762721.399775:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022143360 00000100:00100000:2.0:1789762721.399786:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 70 00000100:00100000:2.0:1789762721.399811:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022142976:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762721.399881:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762721.399926:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:0.0:1789762721.399945:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022143360:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762721.400015:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022142976:12345-0@lo:13:osp-pre-1-0.0 Request processed in 213us (487us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762721.400025:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 70 00000100:00100000:0.0:1789762721.400070:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022143360:12345-0@lo:400:kworker.0 Request processed in 131us (537us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762721.400088:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:2.0:1789762721.400198:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb8ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022143360:0@lo:400:kworker.0 00000100:00100000:0.0:1789762721.400228:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022143104 00000100:00100000:0.0:1789762721.400241:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022143104 00000100:00100000:0.0:1789762721.400350:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 269 00000100:00100000:2.0:1789762721.400357:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb8ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022143232:0@lo:400:kworker.0 00000100:00100000:0.0:1789762721.400370:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022143104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762721.400456:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762721.400472:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022143104:12345-0@lo:400:kworker.0 Request processed in 102us (938us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762721.400510:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 269 00000100:00100000:2.0:1789762721.400584:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022142976:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762721.400783:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb8c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022143488:0@lo:400:kworker.0 00010000:00010000:0.0:1789762721.719299:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762722.743184:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762724.791210:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762725.815165:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762725.986462:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014867584 00000100:00100000:0.0:1789762725.986473:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014867584 00000100:00100000:0.0:1789762725.986535:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 163 00000100:00100000:0.0:1789762725.986549:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014867584:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762725.986618:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014867584:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 75us (227us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762725.986643:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 163 00000100:00100000:0.0:1789762725.988820:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762725.988843:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:0.0:1789762725.988936:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762725.988942:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762725.989056:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762726.007307:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022143744 00000100:00100000:2.0:1789762726.007313:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022143744 00000100:00080000:0.0:1789762726.007347:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762726.007355:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 71 00000100:00100000:2.0:1789762726.007365:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022143744:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762726.007388:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762726.007446:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022143744:12345-0@lo:13:osp-pre-0-0.0 Request processed in 83us (185us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762726.007453:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 71 00000100:00100000:0.0:1789762726.519266:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022144000:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762726.519279:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022144128:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.519402:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762726.519404:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762726.519468:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022144512:0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.519488:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022144128 00000100:00100000:2.0:1789762726.519493:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022144128 00000100:00100000:0.0:1789762726.519502:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762726.519535:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:0.0:1789762726.519545:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022144512 00000100:00100000:0.0:1789762726.519551:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022144512 00000100:00100000:2.0:1789762726.519559:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022144128:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.519596:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 81 00000100:00100000:0.0:1789762726.519611:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022144512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.519617:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022144128:12345-0@lo:400:kworker.0 Request processed in 59us (213us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762726.519628:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:2.0:1789762726.519659:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022144256:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.519670:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022144512:12345-0@lo:400:kworker.0 Request processed in 63us (166us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762726.519679:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 81 00000100:00100000:2.0:1789762726.519688:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762726.519711:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022144256 00000100:00100000:0.0:1789762726.519718:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022144256 00000100:00100000:2.0:1789762726.519736:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022144128:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.519761:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:0.0:1789762726.519769:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022144256:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762726.519782:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762726.519804:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022144256:12345-0@lo:400:kworker.0 Request processed in 36us (116us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762726.519809:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:2.0:1789762726.519821:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022144256:0@lo:400:kworker.0 00000100:00100000:0.0:1789762726.519948:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022143872 00000100:00100000:0.0:1789762726.519962:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022143872 00000100:00100000:0.0:1789762726.520003:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 271 00000100:00100000:0.0:1789762726.520012:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850db850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022143872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.520014:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022144384 00000100:00100000:2.0:1789762726.520020:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022144384 00000100:00100000:0.0:1789762726.520058:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850db850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022143872:12345-0@lo:400:kworker.0 Request processed in 48us (180us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762726.520063:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:0.0:1789762726.520068:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 271 00000100:00100000:2.0:1789762726.520074:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022144384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762726.520144:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022144384:12345-0@lo:400:kworker.0 Request processed in 71us (185us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762726.520152:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00010000:00010000:2.0:1789762726.840190:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762727.863195:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762728.188384:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762728.188431:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014867968 00000100:00100000:2.0:1789762728.188441:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014867968 00000100:00100000:2.0:1789762728.188497:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 147 00000100:00100000:2.0:1789762728.188507:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:10198:x1876702014867968:12345-192.168.203.48@tcp:101:bash.0 00010000:00010000:2.0:1789762728.188524:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762728.188557:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02706 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd242 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.188610:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0270d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.188631:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0270d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.188641:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0270d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762728.188670:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762728.188673:0:5805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762728.188688:0:5805:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0270d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.188707:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02706 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd242 expref: 7 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.188721:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0270d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd242 expref: 6 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762728.188727:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26d00) 00010000:00010000:2.0:1789762728.188730:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762728.188732:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87fdf800, rc 0) 00000100:00100000:2.0:1789762728.188780:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:10198:x1876702014867968:12345-192.168.203.48@tcp:101:bash.0 Request processed in 275us (403us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762728.188785:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 147 00000100:00100000:2.0:1789762728.189878:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762728.189920:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014868096 00000100:00100000:0.0:1789762728.189925:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014868096 00000100:00100000:0.0:1789762728.189956:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 148 00000100:00100000:0.0:1789762728.189968:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:10198:x1876702014868096:12345-192.168.203.48@tcp:101:bash.0 00010000:00010000:0.0:1789762728.189976:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762728.189986:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02714 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd249 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190025:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0271b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190037:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0271b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190045:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0271b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190076:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02722 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190082:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02722 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190123:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02722 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190308:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0271b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190315:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0271b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190320:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0271b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190324:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bce00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762728.190328:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0271b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190333:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0271b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190341:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26d00) 00010000:00010000:0.0:1789762728.190343:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762728.190346:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0271b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190350:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0271b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190357:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02722 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190362:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02722 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190371:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02722 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190374:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bc400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762728.190376:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02722 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190380:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02722 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190385:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28182c63 (ffff981ea0aa3000) 00010000:00010000:0.0:1789762728.190387:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762728.190389:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02722 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190392:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02722 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28182c63 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190412:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02714 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd249 expref: 7 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190419:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26d00) 00010000:00010000:0.0:1789762728.190421:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762728.190423:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02714 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd249 expref: 7 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762728.190428:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bde00, rc 0) 00000100:00100000:0.0:1789762728.190457:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:10198:x1876702014868096:12345-192.168.203.48@tcp:101:bash.0 Request processed in 496us (580us total) trans 0 rc 301/301 00000100:00100000:0.0:1789762728.190470:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 148 00000020:00000080:2.0:1789762728.577124:0:37:0:(genops.c:844:class_export_destroy()) destroying export ffff981d827e4000/f1bd28de-ee78-4d18-bad3-fa7cec968893 for lustre-OST0000 00000020:00000080:2.0:1789762728.577172:0:37:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff981d827e6800 for lustre-OST0000 00000001:02000400:0.0:1789762728.767766:0:13523:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test 77k: check TBF policy with UID/GID/JobID/OPCode expression ========================================================== 16:18:48 (1789762728) 00010000:00010000:0.0:1789762728.887207:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762729.199226:0:13571:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:2.0:1789762729.199234:0:13571:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981e87804000 rate 10000 gen 0 00000100:00100000:2.0:1789762729.199262:0:13571:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:2.0:1789762729.199264:0:13571:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981e87804900 rate 10000 gen 0 00000100:00100000:2.0:1789762729.199418:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:2.0:1789762729.199426:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:2.0:1789762729.199433:0:13571:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:2.0:1789762729.199437:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:2.0:1789762729.199440:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:2.0:1789762729.199444:0:13571:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981e87804f00 rate 20 gen 0 00000100:00100000:2.0:1789762729.199446:0:13571:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:2.0:1789762729.199449:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:2.0:1789762729.199453:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:2.0:1789762729.199457:0:13571:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981e87804300 rate 20 gen 0 00000100:00100000:2.0:1789762729.199545:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:2.0:1789762729.199549:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:2.0:1789762729.199553:0:13571:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:2.0:1789762729.199556:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:2.0:1789762729.199559:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:2.0:1789762729.199561:0:13571:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981e87804400 rate 10 gen 0 00000100:00100000:2.0:1789762729.199563:0:13571:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:2.0:1789762729.199566:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:2.0:1789762729.199568:0:13571:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:2.0:1789762729.199571:0:13571:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981e86cb1d00 rate 10 gen 0 00000100:00100000:2.0:1789762729.212331:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014868224 00000100:00100000:2.0:1789762729.212342:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014868224 00000100:00100000:0.0:1789762729.212395:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014868480 00000100:00100000:2.0:1789762729.212402:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 149 00000100:00100000:0.0:1789762729.212403:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014868480 00000100:00100000:2.0:1789762729.212415:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2363:x1876702014868224:12345-192.168.203.48@tcp:41:mkdir.0 00000100:00100000:0.0:1789762729.212445:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 150 00000004:00000024:2.0:1789762729.212452:0:5804:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:0.0:1789762729.212459:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:12451:x1876702014868480:12345-192.168.203.48@tcp:36:mkdir.0 00010000:00010000:0.0:1789762729.212501:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:2.0:1789762729.212524:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2363:x1876702014868224:12345-192.168.203.48@tcp:41:mkdir.0 Request processed in 111us (231us total) trans 0 rc 0/0 00010000:00010000:0.0:1789762729.212530:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0270d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd242 expref: 7 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.212541:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 149 00010000:00010000:0.0:1789762729.212551:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26d00) 00010000:00010000:0.0:1789762729.212565:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762729.212778:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b02729 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.212792:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b02729 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.212802:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b02729 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.212860:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02730 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.212869:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02730 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.212876:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02730 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762729.214265:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x17:0x0]: returning size 16384 00010000:00010000:0.0:1789762729.214275:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b02729 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.214283:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b02729 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.214290:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b02729 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.214298:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bce00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762729.214302:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b02729 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.214311:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b02729 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.214324:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3000) 00010000:00010000:0.0:1789762729.214328:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762729.214332:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b02729 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.214339:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b02729 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762729.214365:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f0abd8fa reply state 000000009bdb5cd3 transno 4294967413 ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02730 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762729.214374:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762729.214437:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:12451:x1876702014868480:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 1981us (2079us total) trans 4294967413 rc 0/0 00000100:00100000:0.0:1789762729.214455:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 150 00010000:00010000:0.0:1789762729.214959:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02730 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.214969:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02730 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.214975:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02730 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.214981:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bda00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762729.214985:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02730 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.214991:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02730 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.215001:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762729.215005:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762729.215008:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02730 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.215015:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02730 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762729.219487:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014868608 00000100:00100000:0.0:1789762729.219493:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014868608 00000100:00100000:0.0:1789762729.219529:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 151 00000100:00100000:0.0:1789762729.219542:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:12452:x1876702014868608:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762729.219556:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.219578:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02737 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd250 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.219623:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0273e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.219634:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0273e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.219644:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0273e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762729.219666:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762729.219670:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762729.219687:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0273e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.219708:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b02737 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd250 expref: 8 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.219724:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0273e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd250 expref: 7 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.219732:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762729.219735:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762729.219738:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bce00, rc 0) 00000100:00100000:0.0:1789762729.219772:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:12452:x1876702014868608:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 235us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762729.219780:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 151 00000100:00100000:2.0:1789762729.220789:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014868864 00000100:00100000:2.0:1789762729.220795:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014868864 00000100:00100000:2.0:1789762729.220826:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 152 00000100:00100000:2.0:1789762729.220835:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:12452:x1876702014868864:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:2.0:1789762729.220843:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762729.220845:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1789762729.220857:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bce00/0x416193e012b0273e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd250 expref: 7 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.220870:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:2.0:1789762729.220877:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762729.220886:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b02745 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd257 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.220978:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0274c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.220986:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0274c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.220992:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0274c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221002:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b02753 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221007:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b02753 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221012:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b02753 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221121:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b0275a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221130:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b0275a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221137:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b0275a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762729.221143:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x17:0x0]: returning size 16384 00010000:00010000:2.0:1789762729.221153:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b0275a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221157:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b0275a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221162:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b0275a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221166:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fde400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762729.221169:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b0275a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221174:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b0275a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221180:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x0 (ffff981e82a26200) 00010000:00010000:2.0:1789762729.221182:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762729.221185:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b0275a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221197:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b0275a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221222:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0274c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221227:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0274c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221232:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0274c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221235:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fdf800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762729.221237:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0274c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221242:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0274c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221248:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762729.221250:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762729.221252:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0274c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221256:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf800/0x416193e012b0274c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221263:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b02753 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221268:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b02753 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221278:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b02753 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221286:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fdfa00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762729.221289:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b02753 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221294:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b02753 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221299:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e82a26f00) 00010000:00010000:2.0:1789762729.221301:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762729.221303:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b02753 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221307:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfa00/0x416193e012b02753 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221327:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b02745 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd257 expref: 7 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221332:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762729.221334:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762729.221336:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf600/0x416193e012b02745 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd257 expref: 7 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.221343:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87fdf600, rc 0) 00000100:00100000:2.0:1789762729.221373:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:12452:x1876702014868864:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 540us (626us total) trans 0 rc 301/301 00000100:00100000:2.0:1789762729.221387:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 152 00000100:00100000:0.0:1789762729.222170:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014868992 00000100:00100000:0.0:1789762729.222182:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014868992 00000100:00100000:0.0:1789762729.222217:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 153 00000100:00100000:0.0:1789762729.222228:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:12452:x1876702014868992:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762729.222238:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.222252:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bcc00/0x416193e012b02761 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd25e expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.222281:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd600/0x416193e012b02768 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.222291:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd600/0x416193e012b02768 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.222301:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd600/0x416193e012b02768 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762729.222317:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762729.222320:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762729.222331:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd600/0x416193e012b02768 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.222348:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bcc00/0x416193e012b02761 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd25e expref: 8 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.222362:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd600/0x416193e012b02768 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd25e expref: 7 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.222370:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3000) 00010000:00010000:0.0:1789762729.222373:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762729.222375:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bd600, rc 0) 00000100:00100000:0.0:1789762729.222419:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:12452:x1876702014868992:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 194us (282us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762729.222428:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 153 00000100:00100000:0.0:1789762729.223219:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014869248 00000100:00100000:0.0:1789762729.223225:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014869248 00000100:00100000:0.0:1789762729.223256:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 154 00000100:00100000:0.0:1789762729.223266:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:12452:x1876702014869248:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762729.223275:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.223285:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b0276f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd265 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223320:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02776 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223331:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02776 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223341:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02776 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223354:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b0277d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223362:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b0277d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223370:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b0277d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762729.223417:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x17:0x0]: returning size 16384 00010000:00010000:0.0:1789762729.223480:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02776 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223495:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02776 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223502:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02776 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223508:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bc400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762729.223512:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02776 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223519:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02776 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223528:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3000) 00010000:00010000:0.0:1789762729.223532:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762729.223535:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02776 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223542:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b02776 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223554:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b0277d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223561:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b0277d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223567:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b0277d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223572:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bde00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762729.223583:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b0277d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223590:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b0277d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223598:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762729.223601:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762729.223604:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b0277d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223611:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b0277d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223641:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b0276f lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd265 expref: 9 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223652:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3000) 00010000:00010000:0.0:1789762729.223654:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762729.223658:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b0276f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd265 expref: 9 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.223669:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bc800, rc 0) 00000100:00100000:0.0:1789762729.223703:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:12452:x1876702014869248:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 439us (517us total) trans 4294967414 rc 301/301 00000100:00100000:0.0:1789762729.223712:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 154 00000100:00100000:2.0:1789762729.224563:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762729.225425:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b0278b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.225433:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b0278b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.225438:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b0278b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.225443:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812ce00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762729.225445:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b0278b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.225457:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b0278b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.225468:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x0 (ffff981e83a46200) 00010000:00010000:2.0:1789762729.225471:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789762729.225474:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b0278b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x17:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.225479:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ce00/0x416193e012b0278b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x17:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762729.225503:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014869760 00000100:00100000:0.0:1789762729.225510:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014869760 00000100:00100000:0.0:1789762729.225545:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:0.0:1789762729.225554:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:12452:x1876702014869760:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:0.0:1789762729.225675:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:12452:x1876702014869760:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 121us (225us total) trans 4294967416 rc 0/0 00000100:00100000:0.0:1789762729.225684:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:2.0:1789762729.229377:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.229411:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014869888 00000100:00100000:0.0:1789762729.229417:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014869888 00000100:00100000:0.0:1789762729.229457:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 156 00000100:00100000:0.0:1789762729.229467:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:12453:x1876702014869888:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:0.0:1789762729.229478:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.229488:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b02792 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd26c expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229521:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02799 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229531:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02799 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229540:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02799 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229555:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027a0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229563:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229572:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229622:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bcc00/0x416193e012b027a7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229637:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bcc00/0x416193e012b027a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229646:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bcc00/0x416193e012b027a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762729.229664:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x17:0x0]: returning size 16384 00000004:00200000:0.0:1789762729.229667:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762729.229679:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bcc00/0x416193e012b027a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229689:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02799 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229696:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02799 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229703:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02799 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229709:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bde00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762729.229713:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02799 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229721:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02799 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229732:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3000) 00010000:00010000:0.0:1789762729.229735:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762729.229739:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02799 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229756:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b02799 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229768:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229775:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027a0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229783:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027a0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229788:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bc400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762729.229792:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027a0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229799:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027a0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229810:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762729.229813:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762729.229817:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027a0 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229823:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027a0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229849:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b02792 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd26c expref: 12 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229861:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bcc00/0x416193e012b027a7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd26c expref: 11 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.229877:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x0 (ffff981ea0aa3700) 00010000:00010000:0.0:1789762729.229880:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762729.229882:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bcc00, rc 0) 00000100:00100000:0.0:1789762729.229931:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:12453:x1876702014869888:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 466us (555us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762729.229939:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 156 00000100:00100000:2.0:1789762729.230853:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.230904:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014870144 00000100:00100000:0.0:1789762729.230911:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014870144 00000100:00100000:0.0:1789762729.230944:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 157 00000100:00100000:0.0:1789762729.230954:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:12453:x1876702014870144:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:0.0:1789762729.230977:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762729.230988:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bcc00/0x416193e012b027a7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x17:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd26c expref: 11 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.231000:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x0 (ffff981ea0aa3700) 00010000:00010000:0.0:1789762729.231007:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762729.231021:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b027ae lrc: 3/0,1 mode: --/PW res: [0x200000401:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.231030:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b027ae lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.231039:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b027ae lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762729.231185:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000053063ca8 reply state 0000000027ce63e2 transno 4294967417 ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b027ae lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762729.231205:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762729.231211:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x17:0x0]: returning size 16384 00000100:00100000:0.0:1789762729.231258:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:12453:x1876702014870144:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 305us (405us total) trans 4294967417 rc 0/0 00000100:00100000:0.0:1789762729.231265:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 157 00010000:00010000:0.0:1789762729.231716:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b027ae lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.231726:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b027ae lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.231733:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b027ae lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.231740:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bda00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762729.231743:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b027ae lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.231751:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b027ae lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.231760:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762729.231764:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762729.231767:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b027ae lrc: 2/0,0 mode: --/PW res: [0x200000401:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.231774:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bda00/0x416193e012b027ae lrc: 0/0,0 mode: --/PW res: [0x200000401:0x17:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.247848:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.249557:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.249604:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014870528 00000100:00100000:0.0:1789762729.249616:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014870528 00000100:00100000:0.0:1789762729.249652:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 159 00000100:00100000:0.0:1789762729.249662:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:12457:x1876702014870528:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762729.249672:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.249681:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bcc00/0x416193e012b027d1 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd27a expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.249756:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b027d8 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.249767:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b027d8 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.249776:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b027d8 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.249789:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027df lrc: 3/0,1 mode: --/PW res: [0x200000401:0x17:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.249798:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027df lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.249805:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027df lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762729.250330:0:6611:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x9:0x0], index 0: 0 00000004:00080000:0.0:1789762729.250340:0:6611:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x6:0x0], index 1: 0 00010000:00010000:0.0:1789762729.250471:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b027e6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x19:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.250482:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b027e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.250496:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b027e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.250527:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b027d8 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.250534:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b027d8 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.250541:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b027d8 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.250547:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bc800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762729.250551:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b027d8 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.250557:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b027d8 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.250566:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x0 (ffff981e83a46100) 00010000:00010000:0.0:1789762729.250569:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762729.250572:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b027d8 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.250578:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc800/0x416193e012b027d8 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762729.250590:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007dea3652 reply state 00000000ed4ba592 transno 4294967418 ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc400/0x416193e012b027df lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762729.250597:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762729.250624:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bcc00/0x416193e012b027d1 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd27a expref: 15 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.250636:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bde00/0x416193e012b027e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd27a expref: 14 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.250644:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19:0x0].0x0 (ffff981ea0aa3700) 00010000:00010000:0.0:1789762729.250646:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762729.250649:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bde00, rc 0) 00000100:00100000:0.0:1789762729.250687:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:12457:x1876702014870528:12345-192.168.203.48@tcp:101:dd.500 Request processed in 1027us (1131us total) trans 4294967418 rc 0/0 00000100:00100000:0.0:1789762729.250695:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 159 00000100:00100000:2.0:1789762729.253802:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.255007:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762729.255077:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762729.265520:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.265565:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014870912 00000100:00100000:0.0:1789762729.265572:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014870912 00002000:00100000:0.0:1789762729.265604:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x9:0x0] (0->1048575) req@ffff981ebc555f80 x1876702014870912/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762729.265615:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc555f80 with x1876702014870912 ext(0->1048575) 00002000:00010000:0.0:1789762729.265620:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc555f80 00000100:00100000:0.0:1789762729.265643:0:6653:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 3 deadline 0 time 252569546352 state 0x0 ref 1 00000100:00100000:0.0:1789762729.265656:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 3 deadline 252619546352 time 252569546352 state 0x1 ref 1 00000100:00100000:0.0:1789762729.265684:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 2 deadline 252619546352 time 252569590739 state 0x0 ref 1 00000100:00100000:0.0:1789762729.265697:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2363:x1876702014870912:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762729.271642:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.285946:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.288761:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:2363:x1876702014871168:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2630us (2819us total) trans 4294967557 rc 0/0 00000100:00100000:0.0:1789762729.288770:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 2 00002000:00100000:0.0:1789762729.288778:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x9:0x0] (2097152->3145727) req@ffff981ebc4f5c00 x1876702014871168/t4294967557(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762729.288785:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc4f5c00 with x1876702014871168 ext(2097152->3145727) 00002000:00010000:0.0:1789762729.288790:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc4f5c00 00000100:00100000:2.0:1789762729.293265:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.293305:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014871424 00000100:00100000:0.0:1789762729.293311:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014871424 00000100:00100000:0.0:1789762729.293342:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 96 00000100:00100000:0.0:1789762729.293349:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0+-99:2362:x1876702014871424:12345-192.168.203.48@tcp:8:dd.500 00010000:00080000:0.0:1789762729.293371:0:6649:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp t0 exp 0000000000000000 cur 252 last 0 00000020:00000080:0.0:1789762729.293401:0:6649:0:(genops.c:1385:class_connect()) connect: client f1bd28de-ee78-4d18-bad3-fa7cec968893, cookie 0x416193e012b02802 00002000:00100000:0.0:1789762729.293411:0:6649:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli f1bd28de-ee78-4d18-bad3-fa7cec968893/ffff981ebc4cd000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:0.0:1789762729.293426:0:6649:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:0.0:1789762729.293697:0:6649:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:0.0:1789762729.293725:0:6649:0:(import.c:65:import_set_state_nolock()) ffff981ebc4ce000 : changing import state from RECOVER to FULL 00000100:00100000:0.0:1789762729.293763:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2362:x1876702014871424:12345-192.168.203.48@tcp:8:dd.500 Request processed in 415us (503us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762729.293770:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 96 00000100:00100000:2.0:1789762729.294499:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762729.294580:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762729.302028:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.312930:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.312988:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014871808 00000100:00100000:0.0:1789762729.312993:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014871808 00002000:00100000:0.0:1789762729.313014:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x6:0x0] (1048576->2097151) req@ffff981e86007480 x1876702014871808/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762729.313022:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e86007480 with x1876702014871808 ext(1048576->2097151) 00002000:00010000:0.0:1789762729.313031:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e86007480 00000100:00100000:0.0:1789762729.313041:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 252639994145 time 252589994145 state 0x1 ref 3 00000001:00100000:2.0:1789762729.399656:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000100:00100000:2.0:1789762729.401394:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2364:x1876702014871680:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15108us (99362us total) trans 4294967345 rc 0/0 00000100:00100000:2.0:1789762729.401403:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 4 00002000:00100000:2.0:1789762729.401414:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x6:0x0] (0->1048575) req@ffff981ebfdfd180 x1876702014871680/t4294967345(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762729.401422:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfd180 with x1876702014871680 ext(0->1048575) 00002000:00010000:2.0:1789762729.401427:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdfd180 00000100:00100000:2.0:1789762729.403954:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.410609:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.410644:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014872064 00000100:00100000:0.0:1789762729.410650:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014872064 00002000:00100000:0.0:1789762729.410669:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x6:0x0] (3145728->4194303) req@ffff981ead4f6a00 x1876702014872064/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762729.410677:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ead4f6a00 with x1876702014872064 ext(3145728->4194303) 00002000:00010000:0.0:1789762729.410681:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ead4f6a00 00000100:00100000:0.0:1789762729.410690:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 252740156166 time 252690156166 state 0x1 ref 3 00000100:00100000:0.0:1789762729.417274:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.424082:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.430492:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.430544:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014872448 00000100:00100000:0.0:1789762729.430549:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014872448 00002000:00100000:0.0:1789762729.430568:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x9:0x0] (6291456->7340031) req@ffff981ead4f4a80 x1876702014872448/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762729.430575:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ead4f4a80 with x1876702014872448 ext(6291456->7340031) 00002000:00010000:0.0:1789762729.430578:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ead4f4a80 00000100:00100000:0.0:1789762729.430587:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 252740156166 time 252690156166 state 0x1 ref 6 00000100:00100000:2.0:1789762729.437509:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.455504:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.486452:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 252840347891 time 252790347891 state 0x1 ref 8 00000100:00100000:0.0:1789762729.486478:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:2363:x1876702014871936:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762729.489317:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:2363:x1876702014871936:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2847us (85363us total) trans 4294967347 rc 0/0 00000100:00100000:0.0:1789762729.489350:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 6 00002000:00100000:0.0:1789762729.489360:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x6:0x0] (2097152->3145727) req@ffff981ead4f6d80 x1876702014871936/t4294967347(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762729.489369:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ead4f6d80 with x1876702014871936 ext(2097152->3145727) 00002000:00010000:0.0:1789762729.489375:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ead4f6d80 00000100:00100000:2.0:1789762729.498852:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.498931:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014873088 00000100:00100000:0.0:1789762729.498937:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014873088 00002000:00100000:0.0:1789762729.498958:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x6:0x0] (7340032->8388607) req@ffff981ead4f4000 x1876702014873088/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762729.498966:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ead4f4000 with x1876702014873088 ext(7340032->8388607) 00002000:00010000:0.0:1789762729.498970:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ead4f4000 00000100:00100000:0.0:1789762729.498980:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 252840347891 time 252790347891 state 0x1 ref 9 00000100:00100000:2.0:1789762729.499041:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014873344 00000100:00100000:2.0:1789762729.499046:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014873344 00000100:00100000:2.0:1789762729.499081:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 42 00000100:00100000:2.0:1789762729.499111:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:12457:x1876702014873344:12345-192.168.203.48@tcp:35:dd.500 00000100:00100000:2.0:1789762729.499392:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:12457:x1876702014873344:12345-192.168.203.48@tcp:35:dd.500 Request processed in 303us (395us total) trans 4294967419 rc 0/0 00000100:00100000:2.0:1789762729.499399:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 42 00000100:00100000:0.0:1789762729.546452:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014873600 00000100:00100000:0.0:1789762729.546461:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014873600 00000100:00100000:0.0:1789762729.546504:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 160 00000100:00100000:0.0:1789762729.546518:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:12468:x1876702014873600:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762729.546538:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.546555:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b02810 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd28f expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.546651:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b02817 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.546663:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b02817 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.546682:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b02817 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.546692:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc200/0x416193e012b0281e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.546697:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc200/0x416193e012b0281e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.546702:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc200/0x416193e012b0281e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762729.547044:0:8165:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x7:0x0], index 1: 0 00000004:00080000:0.0:1789762729.547050:0:8165:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xa:0x0], index 0: 0 00010000:00010000:0.0:1789762729.547177:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bdc00/0x416193e012b02825 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.547183:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bdc00/0x416193e012b02825 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.547189:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bdc00/0x416193e012b02825 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.547212:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b02817 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.547217:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b02817 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.547222:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b02817 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.547234:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bd000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762729.547237:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b02817 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.547243:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b02817 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.547252:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x0 (ffff981e83a46100) 00010000:00010000:0.0:1789762729.547254:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762729.547257:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b02817 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.547261:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd000/0x416193e012b02817 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762729.547270:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000060cfffb3 reply state 000000002c2242c3 transno 4294967420 ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc200/0x416193e012b0281e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762729.547277:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762729.547297:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bd200/0x416193e012b02810 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd28f expref: 18 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.547305:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bdc00/0x416193e012b02825 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd28f expref: 17 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.547310:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1a:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762729.547312:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762729.547314:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bdc00, rc 0) 00000100:00100000:0.0:1789762729.547354:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:12468:x1876702014873600:12345-192.168.203.48@tcp:101:dd.0 Request processed in 843us (958us total) trans 4294967420 rc 0/0 00000100:00100000:0.0:1789762729.547370:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 160 00010000:00010000:0.0:1789762729.548007:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc200/0x416193e012b0281e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.548017:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc200/0x416193e012b0281e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.548023:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc200/0x416193e012b0281e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.548027:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bc200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762729.548029:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc200/0x416193e012b0281e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.548034:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc200/0x416193e012b0281e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.548042:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x992ad74f (ffff981ea0aa3300) 00010000:00010000:0.0:1789762729.548044:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762729.548047:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc200/0x416193e012b0281e lrc: 2/0,0 mode: --/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.548051:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bc200/0x416193e012b0281e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.554622:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.554646:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014873856 00000100:00100000:0.0:1789762729.554652:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014873856 00000100:00100000:0.0:1789762729.554687:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 99 00000100:00100000:0.0:1789762729.554697:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:12468:x1876702014873856:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762729.554706:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.554722:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001576339a/0x416193e012b0283a lrc: 2/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd29d expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.554747:0:6648:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008743d688/0x416193e012b02833 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd296 expref: 21 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.554755:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762729.554758:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762729.554762:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001576339a/0x416193e012b0283a lrc: 4/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd29d expref: 21 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.554794:0:6648:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008743d688/0x416193e012b02833 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd296 expref: 20 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.554807:0:6648:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008743d688/0x416193e012b02833 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd296 expref: 20 pid: 6881 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762729.554819:0:6648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc4f7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6648:1876702022144640:192.168.203.48@tcp:104: 00000100:00100000:0.0:1789762729.554882:0:6648:0:(client.c:2715:ptlrpc_set_wait()) set ffff981d839b3000 going to sleep for 11 seconds 00000100:00100000:0.0:1789762729.555882:0:6648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc4f7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6648:1876702022144640:192.168.203.48@tcp:104: 00010000:00010000:0.0:1789762729.555914:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001576339a/0x416193e012b0283a lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd29d expref: 20 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.555924:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bc200, rc 0) 00000100:00100000:0.0:1789762729.555953:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:12468:x1876702014873856:12345-192.168.203.48@tcp:101:dd.0 Request processed in 1258us (1335us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762729.555966:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 99 00000100:00100000:2.0:1789762729.562867:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.578773:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014874240 00000100:00100000:0.0:1789762729.578780:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014874240 00000100:00100000:0.0:1789762729.578818:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 102 00000100:00100000:0.0:1789762729.578828:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:12471:x1876702014874240:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762729.578841:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.578854:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fdea70f9/0x416193e012b0284f lrc: 2/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd2b2 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.578872:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x7:0x0].0x0 since=0s v=(2 0) 00010000:00010000:0.0:1789762729.578876:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(3 0) 00010000:00010000:0.0:1789762729.578881:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fdea70f9/0x416193e012b0284f lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2b2 expref: 22 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.578905:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fdea70f9/0x416193e012b0284f lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2b2 expref: 22 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.578915:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bd200, rc 0) 00000100:00100000:2.0:1789762729.578920:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.578955:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:12471:x1876702014874240:12345-192.168.203.48@tcp:101:dd.0 Request processed in 129us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762729.578961:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 102 00000100:00100000:0.0:1789762729.578977:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014874368 00000100:00100000:0.0:1789762729.578980:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014874368 00000100:00100000:0.0:1789762729.579006:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 103 00000100:00100000:0.0:1789762729.579015:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82730380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:12472:x1876702014874368:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762729.579031:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.579038:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000068c3e34/0x416193e012b02856 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd2b9 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.579051:0:6648:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b02848 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ab expref: 16 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.579057:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0xa:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762729.579059:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762729.579063:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000068c3e34/0x416193e012b02856 lrc: 4/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2b9 expref: 16 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.579107:0:6648:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b02848 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ab expref: 16 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.579117:0:6648:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b02848 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ab expref: 16 pid: 6647 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762729.579127:0:6648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc4f7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6648:1876702022144768:192.168.203.48@tcp:104: 00000100:00100000:0.0:1789762729.579181:0:6648:0:(client.c:2715:ptlrpc_set_wait()) set ffff981d839b3000 going to sleep for 11 seconds 00000100:00100000:0.0:1789762729.580881:0:6648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc4f7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6648:1876702022144768:192.168.203.48@tcp:104: 00010000:00010000:0.0:1789762729.580913:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000068c3e34/0x416193e012b02856 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2b9 expref: 16 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.580923:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bd000, rc 0) 00000100:00100000:0.0:1789762729.580957:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82730380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:12472:x1876702014874368:12345-192.168.203.48@tcp:101:dd.0 Request processed in 1945us (2041us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762729.580962:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 103 00000100:00100000:2.0:1789762729.584056:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.585599:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014874752 00000100:00100000:0.0:1789762729.585606:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014874752 00000100:00100000:0.0:1789762729.585634:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 106 00000100:00100000:0.0:1789762729.585641:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf511500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:12476:x1876702014874752:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762729.585649:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.585658:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000a507997/0x416193e012b0286b lrc: 2/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd2ce expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.585673:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x7:0x0].0x0 since=0s v=(3 0) 00010000:00010000:0.0:1789762729.585676:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(4 0) 00010000:00010000:0.0:1789762729.585680:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000a507997/0x416193e012b0286b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 6 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ce expref: 23 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.585688:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000a507997/0x416193e012b0286b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 6 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ce expref: 23 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.585695:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bc000, rc 0) 00000100:00100000:0.0:1789762729.585722:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf511500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:12476:x1876702014874752:12345-192.168.203.48@tcp:101:dd.0 Request processed in 82us (188us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762729.585727:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 106 00000100:00100000:0.0:1789762729.586565:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 252940465506 time 252890465506 state 0x1 ref 8 00000100:00100000:0.0:1789762729.586580:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:2363:x1876702014872192:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762729.589196:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:2363:x1876702014872192:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2617us (171921us total) trans 4294967559 rc 0/0 00000100:00100000:0.0:1789762729.589204:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 8 00002000:00100000:0.0:1789762729.589213:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x9:0x0] (4194304->5242879) req@ffff981ebc4f5f80 x1876702014872192/t4294967559(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762729.589220:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc4f5f80 with x1876702014872192 ext(4194304->5242879) 00002000:00010000:0.0:1789762729.589225:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc4f5f80 00000100:00100000:2.0:1789762729.589453:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.590164:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.590203:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014875136 00000100:00100000:0.0:1789762729.590209:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014875136 00000100:00100000:0.0:1789762729.590248:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 109 00000100:00100000:0.0:1789762729.590258:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:12480:x1876702014875136:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762729.590277:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.590289:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02880 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd2e3 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.590308:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0xa:0x0].0x0 since=0s v=(3 0) 00010000:00010000:0.0:1789762729.590313:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(4 0) 00010000:00010000:0.0:1789762729.590320:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02880 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 6 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2e3 expref: 20 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.590332:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02880 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 6 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2e3 expref: 20 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.590340:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bd400, rc 0) 00000100:00100000:0.0:1789762729.590368:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:12480:x1876702014875136:12345-192.168.203.48@tcp:101:dd.0 Request processed in 113us (206us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762729.590379:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 109 00000100:00100000:2.0:1789762729.590661:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.591118:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.591601:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014875520 00000100:00100000:0.0:1789762729.591606:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014875520 00000100:00100000:0.0:1789762729.591630:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 112 00000100:00100000:0.0:1789762729.591635:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:12481:x1876702014875520:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762729.591641:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762729.591648:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fb240c6a/0x416193e012b02895 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd2f8 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.591660:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0xa:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762729.591663:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762729.591666:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fb240c6a/0x416193e012b02895 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2f8 expref: 23 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.591674:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000fb240c6a/0x416193e012b02895 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2f8 expref: 23 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.591680:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e857bd800, rc 0) 00000100:00100000:0.0:1789762729.591710:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:12481:x1876702014875520:12345-192.168.203.48@tcp:101:dd.0 Request processed in 73us (147us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762729.591718:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 112 00000100:00100000:2.0:1789762729.592680:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.598527:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.598573:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.598645:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014875904 00000100:00100000:0.0:1789762729.598655:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014875904 00002000:00100000:0.0:1789762729.598687:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0xa:0x0] (2097152->3145727) req@ffff981d84543100 x1876702014875904/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762729.598707:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d84543100 with x1876702014875904 ext(2097152->3145727) 00010000:00010000:0.0:1789762729.598728:0:12581:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b02848 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ab expref: 23 pid: 6647 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.598759:0:12581:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b02848 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ab expref: 23 pid: 6647 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762729.598774:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff981d84543100 00000100:00100000:0.0:1789762729.598791:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 3 deadline 252902738202 time 252902638202 state 0x1 ref 2 00000100:00100000:2.0:1789762729.611274:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2364:x1876702014875776:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12446us (12744us total) trans 4294967349 rc 0/0 00000100:00100000:2.0:1789762729.611286:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 0 00002000:00100000:2.0:1789762729.611298:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0x7:0x0] (2097152->3145727) req@ffff981d84543800 x1876702014875776/t4294967349(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762729.611306:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d84543800 with x1876702014875776 ext(2097152->3145727) 00010000:00010000:2.0:1789762729.611315:0:12582:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000008743d688/0x416193e012b02833 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd296 expref: 28 pid: 6881 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.611328:0:12582:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000008743d688/0x416193e012b02833 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd296 expref: 28 pid: 6881 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762729.611342:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981d84543800 00000100:00100000:0.0:1789762729.611609:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2363:x1876702014875904:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12769us (13033us total) trans 4294967560 rc 0/0 00000100:00100000:0.0:1789762729.611621:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1 00000100:00100000:0.0:1789762729.611633:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 1 deadline 252902811634 time 252902731446 state 0x2 ref 0 00002000:00100000:0.0:1789762729.611646:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0xa:0x0] (2097152->3145727) req@ffff981d84543100 x1876702014875904/t4294967560(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762729.611657:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d84543100 with x1876702014875904 ext(2097152->3145727) 00010000:00010000:0.0:1789762729.611666:0:11723:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b02848 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ab expref: 25 pid: 6647 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762729.611680:0:11723:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b02848 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ab expref: 25 pid: 6647 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762729.611688:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff981d84543100 00000100:00100000:2.0:1789762729.615185:0:7595:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014876160 00000100:00100000:2.0:1789762729.615193:0:7595:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014876160 00010000:00010000:2.0:1789762729.615222:0:7595:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b02848 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ab expref: 25 pid: 6647 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.615276:0:7595:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 50 00000100:00100000:2.0:1789762729.615292:0:7595:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2365:x1876702014876160:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762729.615308:0:7595:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1789762729.615322:0:7595:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xa:0x0] updating lvb size from disk: 0 -> 3145728 00002000:00010000:2.0:1789762729.615326:0:7595:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0xa:0x0] updating lvb mtime from disk: 0 -> 1789762729 00002000:00010000:2.0:1789762729.615330:0:7595:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0xa:0x0] updating lvb ctime from disk: 0 -> 1789762729 00010000:00010000:2.0:1789762729.615335:0:7595:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b02848 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ab expref: 25 pid: 6647 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615354:0:7595:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b02848 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ab expref: 25 pid: 6647 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615399:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ccbca09b/0x416193e012b02848 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ab expref: 25 pid: 6647 timeout: 352 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615429:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615441:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615451:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000068c3e34/0x416193e012b02856 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2b9 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615485:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000068c3e34/0x416193e012b02856 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2b9 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615502:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615507:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615513:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0285d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2c0 expref: 24 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615526:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0285d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2c0 expref: 24 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615535:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615543:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615555:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002fb94978/0x416193e012b02864 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2c7 expref: 24 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615577:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002fb94978/0x416193e012b02864 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2c7 expref: 24 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615606:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615614:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615625:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02880 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2e3 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615643:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02880 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2e3 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615673:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615679:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615690:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000005657b01/0x416193e012b02887 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ea expref: 24 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615701:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000005657b01/0x416193e012b02887 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ea expref: 24 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615712:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615717:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615728:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000593beedc/0x416193e012b0288e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2f1 expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615743:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000593beedc/0x416193e012b0288e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2f1 expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615757:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615762:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xa:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762729.615773:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fb240c6a/0x416193e012b02895 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2f8 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615792:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fb240c6a/0x416193e012b02895 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2f8 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762729.615837:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000fb240c6a/0x416193e012b02895 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2f8 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.615859:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc2d2680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:7595:1876702022144896:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762729.615955:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000593beedc/0x416193e012b0288e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2f1 expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.615969:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc2d0a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:7595:1876702022145152:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762729.616007:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000005657b01/0x416193e012b02887 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2ea expref: 24 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.616018:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e8616ed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:7595:1876702022145408:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762729.616061:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02880 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2e3 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.616072:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac58380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:7595:1876702022145664:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762729.616144:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000002fb94978/0x416193e012b02864 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2c7 expref: 24 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.616153:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac59c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:7595:1876702022145792:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762729.616183:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000095389376/0x416193e012b0285d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2c0 expref: 24 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.616191:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc03100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:7595:1876702022145920:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762729.616251:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000068c3e34/0x416193e012b02856 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd2b9 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762729.616265:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d831a9880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:7595:1876702022146176:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762729.616286:0:7595:0:(client.c:2715:ptlrpc_set_wait()) set ffff981eb464a9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762729.617274:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac58380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:7595:1876702022145664:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762729.629227:0:5792:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2366:x1876702014876032:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 13920us (14070us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762729.629239:0:5792:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 51 00000100:00100000:2.0:1789762729.643115:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2363:x1876702014872320:12345-192.168.203.48@tcp:4:dd.500 Request processed in 6465us (219031us total) trans 4294967561 rc 0/0 00000100:00100000:2.0:1789762729.643123:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 9 00002000:00100000:2.0:1789762729.643131:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x9:0x0] (5242880->6291455) req@ffff981ead4f7100 x1876702014872320/t4294967561(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762729.643139:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ead4f7100 with x1876702014872320 ext(5242880->6291455) 00002000:00010000:2.0:1789762729.643144:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ead4f7100 00000100:00100000:0.0:1789762729.686725:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253040625374 time 252990625374 state 0x1 ref 6 00000100:00100000:0.0:1789762729.686749:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2363:x1876702014872448:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762729.719661:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014876288 00000100:00100000:0.0:1789762729.719671:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014876288 00002000:00100000:0.0:1789762729.719713:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x7:0x0] (6291456->7340031) req@ffff981e8600d180 x1876702014876288/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762729.719721:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e8600d180 with x1876702014876288 ext(6291456->7340031) 00002000:00010000:0.0:1789762729.719726:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e8600d180 00000100:00100000:0.0:1789762729.719747:0:12581:0:(nrs_tbf.c:1585:nrs_tbf_cli_findadd()) unknown.0 NRS: TBF class insert: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 3 deadline 0 time 253023653579 state 0x0 ref 1 00000100:00100000:0.0:1789762729.719764:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 3 deadline 253023753579 time 253023653579 state 0x1 ref 1 00000100:00100000:2.0:1789762729.719905:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253023753579 time 253023809051 state 0x0 ref 1 00000100:00100000:2.0:1789762729.719919:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2364:x1876702014876288:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762729.722346:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.723074:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2364:x1876702014876288:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3156us (3507us total) trans 4294967350 rc 0/0 00000100:00100000:2.0:1789762729.723082:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 16 00002000:00100000:2.0:1789762729.723116:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0x7:0x0] (6291456->7340031) req@ffff981e8600d180 x1876702014876288/t4294967350(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762729.723130:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e8600d180 with x1876702014876288 ext(6291456->7340031) 00002000:00010000:2.0:1789762729.723134:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e8600d180 00000100:00100000:2.0:1789762729.735474:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014876672 00000100:00100000:2.0:1789762729.735482:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014876672 00002000:00100000:2.0:1789762729.735512:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xa:0x0] (3145728->4194303) req@ffff981ebfdf9180 x1876702014876672/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762729.735520:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdf9180 with x1876702014876672 ext(3145728->4194303) 00002000:00010000:2.0:1789762729.735525:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdf9180 00000100:00100000:2.0:1789762729.735535:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253026518982 time 253031060792 state 0x0 ref 1 00000100:00100000:2.0:1789762729.735569:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253031160792 time 253031060792 state 0x1 ref 1 00000100:00100000:0.0:1789762729.735588:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.735595:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253031160792 time 253039500211 state 0x0 ref 1 00000100:00100000:2.0:1789762729.735607:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2363:x1876702014876672:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762729.735768:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253039600211 time 253039675084 state 0x0 ref 2 00000100:00100000:2.0:1789762729.735779:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2366:x1876702014876800:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762729.736808:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253090712975 time 253040712975 state 0x1 ref 5 00000100:00100000:0.0:1789762729.736820:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2363:x1876702014872576:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762729.740064:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2363:x1876702014876672:12345-192.168.203.48@tcp:4:dd.0 Request processed in 4455us (4647us total) trans 4294967564 rc 0/0 00000100:00100000:2.0:1789762729.740073:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 19 00002000:00100000:2.0:1789762729.740081:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xa:0x0] (3145728->4194303) req@ffff981ebfdf9180 x1876702014876672/t4294967564(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762729.740113:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdf9180 with x1876702014876672 ext(3145728->4194303) 00002000:00010000:2.0:1789762729.740117:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdf9180 00000100:00100000:0.0:1789762729.741141:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.741168:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.741223:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014877056 00000100:00100000:2.0:1789762729.741229:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014877056 00002000:00100000:2.0:1789762729.741252:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xa:0x0] (4194304->5242879) req@ffff981ebc4f4000 x1876702014877056/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762729.741273:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc4f4000 with x1876702014877056 ext(4194304->5242879) 00002000:00010000:2.0:1789762729.741283:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc4f4000 00000100:00100000:2.0:1789762729.741339:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253039775084 time 253039675084 state 0x1 ref 3 00000100:00100000:2.0:1789762729.741386:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 253045358707 time 253045287823 state 0x0 ref 3 00000100:00100000:2.0:1789762729.741408:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2363:x1876702014877056:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762729.741764:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2366:x1876702014876800:12345-192.168.203.48@tcp:4:dd.0 Request processed in 5986us (6178us total) trans 4294967352 rc 0/0 00000100:00100000:0.0:1789762729.741770:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 20 00002000:00100000:0.0:1789762729.741777:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x7:0x0] (1048576->2097151) req@ffff981ebc4f6a00 x1876702014876800/t4294967352(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762729.741795:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc4f6a00 with x1876702014876800 ext(1048576->2097151) 00002000:00010000:0.0:1789762729.741798:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc4f6a00 00000100:00100000:0.0:1789762729.743174:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2363:x1876702014872576:12345-192.168.203.48@tcp:4:dd.500 Request processed in 6354us (305665us total) trans 4294967565 rc 0/0 00000100:00100000:0.0:1789762729.743188:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 11 00002000:00100000:0.0:1789762729.743196:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x9:0x0] (7340032->8388607) req@ffff981ead4f6680 x1876702014872576/t4294967565(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762729.743203:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ead4f6680 with x1876702014872576 ext(7340032->8388607) 00002000:00010000:0.0:1789762729.743208:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ead4f6680 00000100:00100000:2.0:1789762729.745926:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2364:x1876702014876928:12345-192.168.203.48@tcp:4:dd.0 Request processed in 4547us (4785us total) trans 4294967353 rc 0/0 00000100:00100000:2.0:1789762729.745933:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 21 00000100:00100000:2.0:1789762729.745939:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 253045358707 time 253045287823 state 0x2 ref 0 00002000:00100000:2.0:1789762729.745948:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x7:0x0] (4194304->5242879) req@ffff981ebc4f5180 x1876702014876928/t4294967353(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762729.745955:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc4f5180 with x1876702014876928 ext(4194304->5242879) 00002000:00010000:2.0:1789762729.745959:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc4f5180 00000100:00100000:0.0:1789762729.748908:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762729.754339:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014877312 00000100:00100000:0.0:1789762729.754346:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014877312 00002000:00100000:0.0:1789762729.754368:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0xa:0x0] (5242880->6291455) req@ffff981ebfc03100 x1876702014877312/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762729.754375:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfc03100 with x1876702014877312 ext(5242880->6291455) 00002000:00010000:0.0:1789762729.754378:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfc03100 00000100:00100000:0.0:1789762729.754386:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253045387823 time 253052974692 state 0x0 ref 1 00000100:00100000:0.0:1789762729.754413:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253053074692 time 253052974692 state 0x1 ref 1 00000100:00100000:0.0:1789762729.754431:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253053074692 time 253058340646 state 0x0 ref 1 00000100:00100000:0.0:1789762729.754440:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2363:x1876702014877312:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762729.757860:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014877440 00000100:00100000:2.0:1789762729.757866:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014877440 00002000:00100000:2.0:1789762729.757901:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x7:0x0] (5242880->6291455) req@ffff981e8616ed80 x1876702014877440/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762729.757910:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e8616ed80 with x1876702014877440 ext(5242880->6291455) 00002000:00010000:2.0:1789762729.757914:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e8616ed80 00000100:00100000:2.0:1789762729.757942:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253058440646 time 253058340646 state 0x1 ref 2 00000100:00100000:2.0:1789762729.757968:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253058440646 time 253061876979 state 0x0 ref 2 00000100:00100000:2.0:1789762729.757979:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2366:x1876702014877440:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762729.758298:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2363:x1876702014877312:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3855us (4009us total) trans 4294967568 rc 0/0 00000100:00100000:0.0:1789762729.758308:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 24 00002000:00100000:0.0:1789762729.758318:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0xa:0x0] (5242880->6291455) req@ffff981ebfc03100 x1876702014877312/t4294967568(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762729.758329:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfc03100 with x1876702014877312 ext(5242880->6291455) 00002000:00010000:0.0:1789762729.758334:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfc03100 00000100:00100000:2.0:1789762729.760102:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014877568 00000100:00100000:2.0:1789762729.760107:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014877568 00002000:00100000:2.0:1789762729.760126:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xa:0x0] (7340032->8388607) req@ffff981eb4972300 x1876702014877568/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762729.760134:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4972300 with x1876702014877568 ext(7340032->8388607) 00002000:00010000:2.0:1789762729.760138:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981eb4972300 00000100:00100000:2.0:1789762729.760170:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253061976979 time 253061876979 state 0x1 ref 2 00000100:00100000:2.0:1789762729.760189:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253061976979 time 253064098460 state 0x0 ref 2 00000100:00100000:2.0:1789762729.760199:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4972300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2364:x1876702014877568:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762729.760705:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2366:x1876702014877440:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2727us (2890us total) trans 4294967354 rc 0/0 00000100:00100000:2.0:1789762729.760721:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 25 00002000:00100000:2.0:1789762729.760727:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x7:0x0] (5242880->6291455) req@ffff981e8616ed80 x1876702014877440/t4294967354(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762729.760734:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e8616ed80 with x1876702014877440 ext(5242880->6291455) 00002000:00010000:2.0:1789762729.760737:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e8616ed80 00000100:00100000:2.0:1789762729.766478:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.766519:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014877824 00000100:00100000:2.0:1789762729.766523:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014877824 00002000:00100000:2.0:1789762729.766543:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x7:0x0] (7340032->8388607) req@ffff981d831a9880 x1876702014877824/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/0 e 0 to 0 dl 1789762740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762729.766554:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d831a9880 with x1876702014877824 ext(7340032->8388607) 00002000:00010000:2.0:1789762729.766558:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d831a9880 00000100:00100000:0.0:1789762729.766624:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253064198460 time 253070528469 state 0x0 ref 2 00000100:00100000:0.0:1789762729.766641:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2363:x1876702014877696:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762729.766643:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253070628469 time 253070528469 state 0x1 ref 2 00000100:00100000:2.0:1789762729.766973:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762729.767259:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253070665306 time 253071165184 state 0x0 ref 3 00000100:00100000:2.0:1789762729.767270:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2363:x1876702014877952:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762729.769805:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2363:x1876702014877696:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3168us (3454us total) trans 4294967355 rc 0/0 00000100:00100000:0.0:1789762729.769813:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 27 00002000:00100000:0.0:1789762729.769821:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x7:0x0] (3145728->4194303) req@ffff981ebc2d0a80 x1876702014877696/t4294967355(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762729.769828:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc2d0a80 with x1876702014877696 ext(3145728->4194303) 00002000:00010000:0.0:1789762729.769832:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc2d0a80 00000100:00100000:2.0:1789762729.771762:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2363:x1876702014877952:12345-192.168.203.48@tcp:4:dd.0 Request processed in 4491us (4787us total) trans 4294967570 rc 0/0 00000100:00100000:2.0:1789762729.771770:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 29 00000100:00100000:2.0:1789762729.771778:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253070665306 time 253071165184 state 0x2 ref 0 00002000:00100000:2.0:1789762729.771797:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xa:0x0] (6291456->7340031) req@ffff981d8257e680 x1876702014877952/t4294967570(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762729.771807:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d8257e680 with x1876702014877952 ext(6291456->7340031) 00002000:00010000:2.0:1789762729.771812:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d8257e680 00000100:00100000:2.0:1789762729.786952:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253140848107 time 253090848107 state 0x1 ref 4 00000100:00100000:2.0:1789762729.786976:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2363:x1876702014872704:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762729.789467:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2363:x1876702014872704:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2498us (344177us total) trans 4294967357 rc 0/0 00000100:00100000:2.0:1789762729.789475:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 12 00002000:00100000:2.0:1789762729.789484:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x6:0x0] (4194304->5242879) req@ffff981e8600e680 x1876702014872704/t4294967357(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762729.789492:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e8600e680 with x1876702014872704 ext(4194304->5242879) 00002000:00010000:2.0:1789762729.789497:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e8600e680 00000100:00100000:2.0:1789762729.840196:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:2363:x1876702014872832:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3115us (384692us total) trans 4294967358 rc 0/0 00000100:00100000:2.0:1789762729.840204:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 13 00002000:00100000:2.0:1789762729.840214:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x6:0x0] (5242880->6291455) req@ffff981ead4f5180 x1876702014872832/t4294967358(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762729.840222:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ead4f5180 with x1876702014872832 ext(5242880->6291455) 00002000:00010000:2.0:1789762729.840228:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ead4f5180 00010000:00010000:0.0:1789762729.911168:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762729.937287:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253241063411 time 253241183747 state 0x0 ref 1 00000100:00100000:0.0:1789762729.937310:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:2365:x1876702014873088:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762729.940383:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+45:2365:x1876702014873088:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3078us (441529us total) trans 4294967360 rc 0/0 00000100:00100000:2.0:1789762729.940402:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 15 00000100:00100000:2.0:1789762729.940414:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253241063411 time 253241183747 state 0x2 ref 0 00002000:00100000:2.0:1789762729.940430:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x6:0x0] (7340032->8388607) req@ffff981ead4f4000 x1876702014873088/t4294967360(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:510/0 lens 488/448 e 0 to 0 dl 1789762740 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762729.940438:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ead4f4000 with x1876702014873088 ext(7340032->8388607) 00002000:00010000:2.0:1789762729.940443:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ead4f4000 00000100:00100000:0.0:1789762729.984930:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.023116:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762730.023234:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762730.024797:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.025237:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.025321:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.025806:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.029672:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.075557:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762730.075636:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014879616 00000100:00100000:2.0:1789762730.075645:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014879616 00000100:00100000:2.0:1789762730.075688:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 44 00000100:00100000:2.0:1789762730.075698:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:12519:x1876702014879616:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:2.0:1789762730.075869:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:12519:x1876702014879616:12345-192.168.203.48@tcp:35:dd.0 Request processed in 172us (318us total) trans 4294967423 rc 0/0 00000100:00100000:2.0:1789762730.075874:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 44 00000100:00100000:0.0:1789762730.086443:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.087821:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014880000 00000100:00100000:0.0:1789762730.087827:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014880000 00000100:00100000:0.0:1789762730.087861:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 163 00000100:00100000:0.0:1789762730.087867:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e19880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:12520:x1876702014880000:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:0.0:1789762730.087874:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762730.087893:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3800/0x416193e012b028db lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd329 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762730.087942:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x17:0x0]: returning size 16384 00010000:00010000:0.0:1789762730.088007:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3800/0x416193e012b028db lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd329 expref: 23 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.088021:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3000) 00010000:00010000:0.0:1789762730.088024:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762730.088028:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3800/0x416193e012b028db lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd329 expref: 23 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.088036:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebf4f3800, rc 0) 00000100:00100000:0.0:1789762730.088065:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e19880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:12520:x1876702014880000:12345-192.168.203.48@tcp:101:rm.0 Request processed in 199us (293us total) trans 4294967424 rc 301/301 00000100:00100000:0.0:1789762730.088080:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 163 00000100:00100000:0.0:1789762730.089376:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762730.092436:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014880384 00000100:00100000:2.0:1789762730.092441:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014880384 00000100:00100000:2.0:1789762730.092472:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 46 00000100:00100000:2.0:1789762730.092482:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b18050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:12520:x1876702014880384:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:2.0:1789762730.092576:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b18050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:12520:x1876702014880384:12345-192.168.203.48@tcp:35:rm.0 Request processed in 97us (187us total) trans 4294967425 rc 0/0 00000100:00100000:2.0:1789762730.092582:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 46 00000100:00100000:0.0:1789762730.093420:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.094535:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762730.095513:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.095542:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014881024 00000100:00100000:0.0:1789762730.095548:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014881024 00000100:00100000:0.0:1789762730.095573:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 166 00000100:00100000:0.0:1789762730.095578:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:12520:x1876702014881024:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:0.0:1789762730.095600:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1789762730.095612:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e857bdc00/0x416193e012b02825 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd28f expref: 25 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.095623:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1a:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762730.095632:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfec9e00/0x416193e012b028d4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x17:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd322 expref: 24 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.095641:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x0 (ffff981e83a46100) 00010000:00010000:0.0:1789762730.095644:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762730.095656:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2600/0x416193e012b028fe lrc: 3/0,1 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.095666:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2600/0x416193e012b028fe lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.095675:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2600/0x416193e012b028fe lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.095690:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3e00/0x416193e012b02905 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.095703:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3e00/0x416193e012b02905 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.095708:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3e00/0x416193e012b02905 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.095761:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3000/0x416193e012b0290c lrc: 3/0,1 mode: --/EX res: [0x200000401:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.095766:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3000/0x416193e012b0290c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.095771:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3000/0x416193e012b0290c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762730.096381:0:8165:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].6, 64 off8576 00000004:00080000:0.0:1789762730.096390:0:8165:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 253400302301ns, next at 254400300251ns, rc = 0 00000040:00080000:0.0:1789762730.096472:0:8165:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].9, 64 off8768 00000004:00080000:0.0:1789762730.096478:0:8165:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 253400390721ns, next at 254400389633ns, rc = 0 00010000:00010000:0.0:1789762730.096518:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2a00/0x416193e012b02913 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096525:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2a00/0x416193e012b02913 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096531:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2a00/0x416193e012b02913 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096536:0:8165:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2a00/0x416193e012b02913 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096549:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2a00/0x416193e012b02913 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096554:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2a00/0x416193e012b02913 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096568:0:8165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762730.096578:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3000/0x416193e012b0290c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096583:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3000/0x416193e012b0290c lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096588:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3000/0x416193e012b0290c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.096588:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762730.096594:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f3000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762730.096598:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3000/0x416193e012b0290c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.096601:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2a00/0x416193e012b02913 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096605:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3000/0x416193e012b0290c lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.096614:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762730.096621:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1a:0x0].0x0 (ffff981ea0aa3c00) 00010000:00010000:2.0:1789762730.096621:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2a00/0x416193e012b02913 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096628:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762730.096633:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3000/0x416193e012b0290c lrc: 2/0,0 mode: --/EX res: [0x200000401:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.096635:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2a00/0x416193e012b02913 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096649:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3000/0x416193e012b0290c lrc: 0/0,0 mode: --/EX res: [0x200000401:0x1a:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.096655:0:5793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1a:0x0].0x0 (ffff981ea0aa3c00) 00010000:00010000:2.0:1789762730.096659:0:5793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789762730.096662:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2a00/0x416193e012b02913 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.096667:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2a00/0x416193e012b02913 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096669:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2600/0x416193e012b028fe lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.096679:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762730.096683:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762730.096685:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762730.096689:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2600/0x416193e012b028fe lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096697:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2600/0x416193e012b028fe lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096702:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762730.096704:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2600/0x416193e012b028fe lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096709:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2600/0x416193e012b028fe lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096716:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x0 (ffff981e83a46100) 00010000:00010000:0.0:1789762730.096718:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762730.096720:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2600/0x416193e012b028fe lrc: 2/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.096724:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2600/0x416193e012b028fe lrc: 0/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762730.096733:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001acc408c reply state 00000000eafbcaf2 transno 4294967427 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3e00/0x416193e012b02905 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762730.096739:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762730.096774:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:12520:x1876702014881024:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1195us (1261us total) trans 4294967427 rc 0/0 00000100:00100000:0.0:1789762730.096780:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 166 00000100:00100000:2.0:1789762730.097791:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762730.098482:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762730.098488:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c000/0x416193e012b0292f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x19:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.098511:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762730.098517:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c000/0x416193e012b0292f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x19:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.098526:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c000/0x416193e012b0292f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x19:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.098559:0:8466:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x19:0x0].0x0 (ffff981e83a46500) 00010000:00010000:0.0:1789762730.098563:0:8466:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762730.098567:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c000/0x416193e012b0292f lrc: 1/0,0 mode: --/PW res: [0x200000401:0x19:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.098579:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812c000/0x416193e012b0292f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x19:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.098610:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762730.098612:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762730.098615:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762730.099148:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d400/0x416193e012b02921 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x17:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.099155:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d400/0x416193e012b02921 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.099160:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d400/0x416193e012b02921 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.099163:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762730.099172:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d400/0x416193e012b02921 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.099176:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d400/0x416193e012b02921 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x17:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.099184:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x77289514 (ffff981e83a46600) 00010000:00010000:2.0:1789762730.099186:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762730.099188:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d400/0x416193e012b02921 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x17:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.099192:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d400/0x416193e012b02921 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x17:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762730.100573:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.101374:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.378689:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762730.378736:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014881920 00000100:00100000:2.0:1789762730.378745:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014881920 00000100:00100000:2.0:1789762730.378796:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 169 00000100:00100000:2.0:1789762730.378805:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:12531:x1876702014881920:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:2.0:1789762730.378825:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762730.378843:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde800/0x416193e012b0294b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd337 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.378907:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde000/0x416193e012b02952 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.378919:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde000/0x416193e012b02952 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.378927:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde000/0x416193e012b02952 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762730.378965:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762730.378968:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762730.378982:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde000/0x416193e012b02952 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.379001:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde800/0x416193e012b0294b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd337 expref: 25 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.379013:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde000/0x416193e012b02952 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd337 expref: 24 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.379020:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26200) 00010000:00010000:2.0:1789762730.379022:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762730.379025:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87fde000, rc 0) 00000100:00100000:2.0:1789762730.379074:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:12531:x1876702014881920:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 271us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762730.379080:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 169 00000100:00100000:2.0:1789762730.380270:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762730.381989:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d200/0x416193e012b02960 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.381998:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d200/0x416193e012b02960 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.382003:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d200/0x416193e012b02960 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.382007:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762730.382016:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d200/0x416193e012b02960 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.382021:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d200/0x416193e012b02960 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.382030:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e83a46b00) 00010000:00010000:2.0:1789762730.382032:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762730.382034:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d200/0x416193e012b02960 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.382039:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812d200/0x416193e012b02960 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762730.387500:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.387549:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014882304 00000100:00100000:0.0:1789762730.387556:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014882304 00000100:00100000:0.0:1789762730.387602:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 171 00000100:00100000:0.0:1789762730.387611:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:12532:x1876702014882304:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762730.387627:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762730.387639:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2200/0x416193e012b02967 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd33e expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.387679:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3c00/0x416193e012b0296e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.387699:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3c00/0x416193e012b0296e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.387710:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3c00/0x416193e012b0296e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762730.387738:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762730.387741:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762730.387755:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3c00/0x416193e012b0296e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.387775:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2200/0x416193e012b02967 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd33e expref: 26 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.387787:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3c00/0x416193e012b0296e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd33e expref: 25 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.387793:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762730.387795:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762730.387797:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebf4f3c00, rc 0) 00000100:00100000:0.0:1789762730.387839:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:12532:x1876702014882304:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 229us (343us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762730.387845:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 171 00000100:00100000:2.0:1789762730.389125:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014882560 00000100:00100000:2.0:1789762730.389131:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014882560 00000100:00100000:2.0:1789762730.389169:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 172 00000100:00100000:2.0:1789762730.389178:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e18380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:12532:x1876702014882560:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:2.0:1789762730.389186:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762730.389188:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1789762730.389202:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f3c00/0x416193e012b0296e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd33e expref: 25 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389216:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:2.0:1789762730.389223:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762730.389231:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfc00/0x416193e012b02975 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd345 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389310:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b0297c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389324:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b0297c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389331:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b0297c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389342:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02983 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389347:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02983 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389352:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02983 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389403:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b0298a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389412:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b0298a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389420:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b0298a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762730.389430:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1b:0x0]: returning size 16384 00010000:00010000:2.0:1789762730.389442:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b0298a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389449:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b0298a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389453:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b0298a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389465:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fdea00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762730.389468:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b0298a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389473:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b0298a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389482:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1b:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762730.389484:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762730.389486:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b0298a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389490:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b0298a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389507:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b0297c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389511:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b0297c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389515:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b0297c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389519:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fdf200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762730.389521:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b0297c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389524:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b0297c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389538:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26f00) 00010000:00010000:2.0:1789762730.389539:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762730.389541:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b0297c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389546:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b0297c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389554:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02983 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389558:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02983 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389563:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02983 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389567:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fde200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762730.389569:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02983 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389573:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02983 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389578:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e82a26800) 00010000:00010000:2.0:1789762730.389579:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762730.389581:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02983 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389584:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde200/0x416193e012b02983 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389613:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfc00/0x416193e012b02975 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd345 expref: 25 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389619:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26f00) 00010000:00010000:2.0:1789762730.389621:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762730.389624:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfc00/0x416193e012b02975 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd345 expref: 25 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.389633:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87fdfc00, rc 0) 00000100:00100000:2.0:1789762730.389676:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e18380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:12532:x1876702014882560:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 499us (615us total) trans 0 rc 301/301 00000100:00100000:2.0:1789762730.389685:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 172 00000100:00100000:2.0:1789762730.390504:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762730.393434:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.393472:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014883200 00000100:00100000:0.0:1789762730.393479:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014883200 00000100:00100000:0.0:1789762730.393514:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 175 00000100:00100000:0.0:1789762730.393523:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82731880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:12532:x1876702014883200:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:0.0:1789762730.393555:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2200/0x416193e012b029bb lrc: 3/0,1 mode: --/PW res: [0x200000401:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.393569:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2200/0x416193e012b029bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.393575:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2200/0x416193e012b029bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762730.393713:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002944e01b reply state 0000000047c2d23b transno 4294967433 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2200/0x416193e012b029bb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762730.393723:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762730.393730:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1b:0x0]: returning size 16384 00000100:00100000:0.0:1789762730.393775:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82731880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:12532:x1876702014883200:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 253us (341us total) trans 4294967433 rc 0/0 00000100:00100000:0.0:1789762730.393780:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 175 00000100:00100000:2.0:1789762730.394467:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.394507:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014883456 00000100:00100000:0.0:1789762730.394512:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014883456 00000100:00100000:0.0:1789762730.394539:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 48 00000100:00100000:0.0:1789762730.394546:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82730380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:12532:x1876702014883456:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:0.0:1789762730.394637:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82730380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:12532:x1876702014883456:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 91us (175us total) trans 4294967434 rc 0/0 00000100:00100000:0.0:1789762730.394642:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 48 00000100:00100000:2.0:1789762730.398627:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762730.400985:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfec8400/0x416193e012b029de lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.400993:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfec8400/0x416193e012b029de lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.400999:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfec8400/0x416193e012b029de lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.401008:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfec8400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762730.401011:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfec8400/0x416193e012b029de lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.401015:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfec8400/0x416193e012b029de lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x1b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.401023:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1b:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:0.0:1789762730.401025:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762730.401027:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfec8400/0x416193e012b029de lrc: 2/0,0 mode: --/PW res: [0x200000401:0x1b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.401031:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfec8400/0x416193e012b029de lrc: 0/0,0 mode: --/PW res: [0x200000401:0x1b:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762730.424289:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014883968 00000100:00100000:2.0:1789762730.424297:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014883968 00000100:00100000:2.0:1789762730.424346:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 178 00000100:00100000:2.0:1789762730.424358:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf565f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:12539:x1876702014883968:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:2.0:1789762730.424381:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762730.424399:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b029e5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd361 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424458:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b029ec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424480:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b029ec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424490:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b029ec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424500:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfe00/0x416193e012b029f3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424505:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfe00/0x416193e012b029f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424509:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfe00/0x416193e012b029f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424594:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b029fa lrc: 3/1,0 mode: --/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424604:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b029fa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424612:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b029fa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762730.424629:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1b:0x0]: returning size 16384 00000004:00200000:2.0:1789762730.424632:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762730.424646:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b029fa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424658:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b029ec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424677:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b029ec lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424685:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b029ec lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424692:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fdea00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762730.424695:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b029ec lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424701:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b029ec lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424711:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46000) 00010000:00010000:2.0:1789762730.424713:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762730.424716:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b029ec lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424720:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdea00/0x416193e012b029ec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424728:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfe00/0x416193e012b029f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424732:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfe00/0x416193e012b029f3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424736:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfe00/0x416193e012b029f3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424740:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fdfe00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762730.424746:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfe00/0x416193e012b029f3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424750:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfe00/0x416193e012b029f3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424755:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e82a26f00) 00010000:00010000:2.0:1789762730.424757:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762730.424758:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfe00/0x416193e012b029f3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424762:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdfe00/0x416193e012b029f3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424784:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fdf200/0x416193e012b029e5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd361 expref: 31 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424793:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87fde400/0x416193e012b029fa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd361 expref: 30 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.424798:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1b:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762730.424800:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789762730.424802:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87fde400, rc 0) 00000100:00100000:2.0:1789762730.424848:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf565f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:12539:x1876702014883968:12345-192.168.203.48@tcp:101:dd.500 Request processed in 494us (614us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762730.424853:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 178 00000100:00100000:2.0:1789762730.426054:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762730.430606:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.430652:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014884352 00000100:00100000:0.0:1789762730.430658:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014884352 00000100:00100000:0.0:1789762730.430694:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253241063411 time 253241183747 state 0x0 ref 1 00000100:00100000:0.0:1789762730.430704:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253291183747 time 253241183747 state 0x1 ref 1 00000100:00100000:0.0:1789762730.430725:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 2 deadline 253291183747 time 253734634309 state 0x0 ref 1 00000100:00100000:0.0:1789762730.430735:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2366:x1876702014884352:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762730.430761:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789762730.430766:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e6f1ce0/0x416193e012b02a24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.430786:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=253s v=(0 0) 00010000:00010000:0.0:1789762730.430790:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762730.430795:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e6f1ce0/0x416193e012b02a24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.430807:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000e6f1ce0/0x416193e012b02a24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762730.450951:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967568 is committed 00000001:00080000:0.0:1789762730.450978:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967571 is committed 00000100:00100000:0.0:1789762730.462146:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762730.474463:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967572 is committed 00000100:00100000:2.0:1789762730.476785:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.476832:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014884608 00000100:00100000:0.0:1789762730.476838:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014884608 00000100:00100000:0.0:1789762730.476867:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 1 deadline 253784634309 time 253766180635 state 0x0 ref 1 00000100:00100000:0.0:1789762730.476877:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 1 deadline 253816180635 time 253766180635 state 0x1 ref 1 00000100:00100000:0.0:1789762730.476911:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253816180635 time 253780820151 state 0x0 ref 1 00000100:00100000:0.0:1789762730.476920:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf511500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014884608:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762730.476945:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 2097152, mtime: 0x6aad9caa, atime: 0x0, ctime: 0x6aad9caa, blocks: 0x1007 00010000:00010000:0.0:1789762730.476950:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02a32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.476968:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=253s v=(0 0) 00010000:00010000:0.0:1789762730.476971:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762730.476975:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02a32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.476983:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02a32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762730.488629:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967573 is committed 00000100:00100000:2.0:1789762730.490760:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762730.526997:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253830820151 time 253830898944 state 0x0 ref 1 00000100:00100000:2.0:1789762730.527012:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014884736:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762730.527045:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 3145728, mtime: 0x6aad9caa, atime: 0x0, ctime: 0x6aad9caa, blocks: 0x1807 00010000:00010000:2.0:1789762730.527051:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02a39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.527062:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=253s v=(0 0) 00010000:00010000:2.0:1789762730.527066:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762730.527069:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02a39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762730.527079:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02a39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762730.538907:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967574 is committed 00002000:00010000:0.0:1789762730.538914:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:0.0:1789762730.538919:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:0.0:1789762730.538928:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02a39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.538937:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02a39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.538943:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02a39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.538948:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fdee00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762730.538951:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02a39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.538958:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02a39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.538980:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02a39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.538985:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02a39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762730.539034:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014884736:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12023us (48273us total) trans 4294967574 rc 0/0 00000100:00100000:0.0:1789762730.539041:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 34 00000100:00100000:0.0:1789762730.539048:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253830820151 time 253830898944 state 0x2 ref 0 00000001:00080000:2.0:1789762730.589675:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967575 is committed 00002000:00010000:0.0:1789762730.589681:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:0.0:1789762730.589688:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:0.0:1789762730.589702:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f99e975/0x416193e012b02a40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.589717:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009f99e975/0x416193e012b02a40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.589728:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009f99e975/0x416193e012b02a40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.589734:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfec9a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762730.589738:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009f99e975/0x416193e012b02a40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.589755:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009f99e975/0x416193e012b02a40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.589770:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009f99e975/0x416193e012b02a40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.589776:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009f99e975/0x416193e012b02a40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762730.589844:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014884864:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12763us (48656us total) trans 4294967575 rc 0/0 00000100:00100000:0.0:1789762730.589852:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 35 00000100:00100000:0.0:1789762730.589862:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253880898944 time 253880951825 state 0x2 ref 0 00000100:00100000:0.0:1789762730.627132:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253930951825 time 253931028856 state 0x0 ref 1 00000100:00100000:0.0:1789762730.627153:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014884992:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762730.627202:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 5242880, mtime: 0x6aad9caa, atime: 0x0, ctime: 0x6aad9caa, blocks: 0x2807 00010000:00010000:0.0:1789762730.627209:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.627222:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=253s v=(0 0) 00010000:00010000:0.0:1789762730.627226:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762730.627231:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.627252:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762730.637957:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 5242880 -> 6291456 00000001:00080000:2.0:1789762730.637958:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967576 is committed 00002000:00010000:0.0:1789762730.637963:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:0.0:1789762730.637974:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.637983:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.637990:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.637995:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f3a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762730.637999:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.638005:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.638018:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.638023:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762730.638118:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014884992:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10941us (45653us total) trans 4294967576 rc 0/0 00000100:00100000:0.0:1789762730.638126:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 36 00000100:00100000:0.0:1789762730.638134:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253930951825 time 253931028856 state 0x2 ref 0 00000100:00100000:0.0:1789762730.677215:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253981028856 time 253981117713 state 0x0 ref 1 00000100:00100000:0.0:1789762730.677243:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702014885120:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762730.677277:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 6291456, mtime: 0x6aad9caa, atime: 0x0, ctime: 0x6aad9caa, blocks: 0x3007 00010000:00010000:0.0:1789762730.677283:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.677294:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=253s v=(0 0) 00010000:00010000:0.0:1789762730.677297:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762730.677301:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.677312:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762730.688443:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967577 is committed 00002000:00010000:0.0:1789762730.688449:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:0.0:1789762730.688456:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:0.0:1789762730.688467:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.688494:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.688504:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.688513:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f3a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762730.688518:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.688528:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.688544:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.688549:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762730.688600:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702014885120:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11359us (48177us total) trans 4294967577 rc 0/0 00000100:00100000:0.0:1789762730.688607:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 37 00000100:00100000:0.0:1789762730.688615:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 253981028856 time 253981117713 state 0x2 ref 0 00000100:00100000:0.0:1789762730.727301:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 254031117713 time 254031192849 state 0x0 ref 1 00000100:00100000:0.0:1789762730.727330:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702014885248:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762730.727379:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 7340032, mtime: 0x6aad9caa, atime: 0x0, ctime: 0x6aad9caa, blocks: 0x3807 00010000:00010000:0.0:1789762730.727386:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.727398:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=254s v=(0 0) 00010000:00010000:0.0:1789762730.727402:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762730.727406:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.727418:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762730.738208:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 7340032 -> 8388608 00000001:00080000:2.0:1789762730.738208:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967578 is committed 00002000:00010000:0.0:1789762730.738215:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:0.0:1789762730.738226:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.738236:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.738242:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.738247:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f3a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762730.738257:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.738263:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.738277:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.738283:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762730.738339:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702014885248:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11012us (47495us total) trans 4294967578 rc 0/0 00000100:00100000:0.0:1789762730.738347:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 38 00000100:00100000:0.0:1789762730.738357:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 254031117713 time 254031192849 state 0x2 ref 0 00000100:00100000:2.0:1789762730.740650:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.777359:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 254081192849 time 254081262905 state 0x0 ref 1 00000100:00100000:0.0:1789762730.777374:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014885376:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762730.777408:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 8388608, mtime: 0x6aad9caa, atime: 0x0, ctime: 0x6aad9caa, blocks: 0x4007 00010000:00010000:0.0:1789762730.777414:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.777426:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=254s v=(0 0) 00010000:00010000:0.0:1789762730.777430:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762730.777440:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.777450:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097ec75fa/0x416193e012b02a5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762730.788220:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967579 is committed 00000100:00100000:2.0:1789762730.790959:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762730.827437:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 254131262905 time 254131337677 state 0x0 ref 1 00000100:00100000:0.0:1789762730.827454:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014885504:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762730.827496:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 9437184, mtime: 0x6aad9caa, atime: 0x0, ctime: 0x6aad9caa, blocks: 0x4807 00010000:00010000:0.0:1789762730.827502:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b322300e/0x416193e012b02a63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.827515:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=254s v=(0 0) 00010000:00010000:0.0:1789762730.827518:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762730.827523:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b322300e/0x416193e012b02a63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762730.827534:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b322300e/0x416193e012b02a63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762730.839067:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967580 is committed 00010000:00010000:0.0:1789762730.935327:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762730.991633:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762731.027729:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:0.0:1789762731.037847:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967584 is committed 00000001:00080000:0.0:1789762731.089424:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967585 is committed 00000100:00100000:2.0:1789762731.106027:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762731.106060:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762731.106113:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762731.106152:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014886400 00000100:00100000:2.0:1789762731.106153:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762731.106160:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014886400 00000100:00080000:2.0:1789762731.106201:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762731.106211:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 272 00000100:00100000:2.0:1789762731.106218:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014886656 00000100:00100000:2.0:1789762731.106225:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014886656 00000100:00100000:0.0:1789762731.106235:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014886400:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762731.106269:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 82 00000100:00100000:0.0:1789762731.106280:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014886400:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 49us (192us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762731.106281:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e90598a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2365:x1876702014886656:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762731.106295:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 272 00000100:00100000:2.0:1789762731.106323:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e90598a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2365:x1876702014886656:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 45us (173us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762731.106330:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 82 00000100:00080000:2.0:1789762731.106352:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000004:00080000:2.0:1789762731.127197:0:6868:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 254431108081ns, next at 255431105501ns, rc = 0 00000100:00100000:0.0:1789762731.127219:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e90598700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022146688:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762731.127256:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762731.127293:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022146688 00000100:00100000:2.0:1789762731.127298:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022146688 00000100:00100000:2.0:1789762731.127340:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 73 00000100:00100000:2.0:1789762731.127349:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022146688:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762731.127362:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 97058816 pending 0 free 3835691008 avail 3825205248 00000100:00100000:2.0:1789762731.127394:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022146688:12345-0@lo:13:osp-pre-0-0.0 Request processed in 49us (139us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762731.127400:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 73 00000100:00100000:0.0:1789762731.127440:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e90598700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022146688:0@lo:13:osp-pre-0-0.0 00000001:00080000:0.0:1789762731.242550:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967588 is committed 00002000:00010000:0.0:1789762731.389147:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:0.0:1789762731.389153:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:0.0:1789762731.389164:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001ec40dc7/0x416193e012b02ab0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762731.389178:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ec40dc7/0x416193e012b02ab0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762731.389188:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ec40dc7/0x416193e012b02ab0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762731.389196:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfec8c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762731.389202:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ec40dc7/0x416193e012b02ab0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762731.389214:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ec40dc7/0x416193e012b02ab0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762731.389244:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ec40dc7/0x416193e012b02ab0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762731.389250:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ec40dc7/0x416193e012b02ab0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762731.389325:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014887424:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11270us (48168us total) trans 4294967591 rc 0/0 00000100:00100000:0.0:1789762731.389333:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 51 00000100:00100000:0.0:1789762731.389343:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 254681893793 time 254681942748 state 0x2 ref 0 00000100:00100000:0.0:1789762731.478197:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 254782024596 time 254782096485 state 0x0 ref 1 00000100:00100000:0.0:1789762731.478216:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014887680:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762731.478257:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 23068672, mtime: 0x6aad9cab, atime: 0x0, ctime: 0x6aad9cab, blocks: 0xb007 00010000:00010000:0.0:1789762731.478263:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000000e57b25/0x416193e012b02abe lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762731.478276:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=254s v=(0 0) 00010000:00010000:0.0:1789762731.478279:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762731.478284:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000000e57b25/0x416193e012b02abe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762731.478295:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000000e57b25/0x416193e012b02abe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762731.596349:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762731.639602:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022146816:0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.639703:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762731.639791:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8a971880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022147328:0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.639929:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762731.639997:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022147328 00000100:00100000:2.0:1789762731.640007:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022147328 00000100:00100000:2.0:1789762731.640060:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:2.0:1789762731.640085:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a972d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022147072:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.640188:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a972d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022147072:12345-0@lo:400:kworker.0 Request processed in 113us (585us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762731.640200:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:2.0:1789762731.640267:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022146816 00000100:00100000:2.0:1789762731.640277:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022146816 00000100:00100000:2.0:1789762731.640361:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 273 00000100:00100000:2.0:1789762731.640385:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022146816:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.640508:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022146816:12345-0@lo:400:kworker.0 Request processed in 126us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762731.640525:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 273 00000100:00100000:2.0:1789762731.640770:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022146816:0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.640799:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8a971880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022147328:0@lo:400:kworker.0 00000100:00100000:2.0:1789762731.648613:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762731.959212:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762732.003762:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762732.102189:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967605 is committed 00000001:00080000:2.0:1789762732.150539:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967606 is committed 00000001:00080000:2.0:1789762732.202745:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967607 is committed 00000100:00100000:0.0:1789762732.206949:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014889600 00000100:00100000:0.0:1789762732.206962:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014889600 00000100:00100000:0.0:1789762732.207027:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 255483355831 time 255483510559 state 0x0 ref 1 00000100:00100000:0.0:1789762732.207047:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 255533510559 time 255483510559 state 0x1 ref 1 00000001:00080000:2.0:1789762732.251379:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967608 is committed 00000001:00080000:2.0:1789762732.301754:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967609 is committed 00000001:00080000:2.0:1789762732.351277:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967610 is committed 00000001:00080000:2.0:1789762732.401304:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967611 is committed 00000100:00100000:2.0:1789762732.406146:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762732.452360:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967612 is committed 00000001:00080000:2.0:1789762732.502357:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967613 is committed 00000001:00080000:2.0:1789762732.552329:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967614 is committed 00000100:00100000:2.0:1789762732.656177:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762732.856947:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762732.906896:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762732.956650:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762732.983286:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762733.057309:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762733.210620:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762733.302921:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967629 is committed 00000001:00080000:0.0:1789762733.352544:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967630 is committed 00000100:00100000:2.0:1789762733.356446:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762733.398881:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967631 is committed 00000100:00100000:2.0:1789762733.402521:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762733.447337:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967632 is committed 00000001:00080000:0.0:1789762733.495286:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967633 is committed 00000001:00080000:0.0:1789762733.546330:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967634 is committed 00000100:00100000:2.0:1789762733.598838:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762733.648636:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762733.698188:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762733.751350:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762733.846778:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967640 is committed 00000100:00100000:2.0:1789762733.849665:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762733.898004:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967641 is committed 00002000:00010000:0.0:1789762733.951947:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:0.0:1789762733.951956:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 145419 -> 147467 00010000:00010000:0.0:1789762733.951971:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000411f31ea/0x416193e012b02c15 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762733.951986:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000411f31ea/0x416193e012b02c15 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762733.951998:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000411f31ea/0x416193e012b02c15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762733.952007:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfec8e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762733.952026:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000411f31ea/0x416193e012b02c15 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762733.952039:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000411f31ea/0x416193e012b02c15 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762733.952060:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000411f31ea/0x416193e012b02c15 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762733.952070:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000411f31ea/0x416193e012b02c15 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762733.952211:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014893952:12345-192.168.203.48@tcp:4:dd.500 Request processed in 18793us (51377us total) trans 4294967642 rc 0/0 00000100:00100000:0.0:1789762733.952221:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 102 00000100:00100000:0.0:1789762733.952253:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257237172870 time 257237281566 state 0x2 ref 0 00002000:00010000:0.0:1789762734.004183:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:0.0:1789762734.004189:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 147467 -> 149515 00010000:00010000:0.0:1789762734.004203:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02c1c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.004214:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02c1c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.004222:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02c1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.004241:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfec9e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.004245:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02c1c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.004255:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02c1c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.004278:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02c1c lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.004289:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02c1c lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.004381:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014894080:12345-192.168.203.48@tcp:4:dd.500 Request processed in 20880us (48622us total) trans 4294967643 rc 0/0 00000100:00100000:0.0:1789762734.004395:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 103 00000100:00100000:0.0:1789762734.004412:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257287281566 time 257287375879 state 0x2 ref 0 00010000:00010000:0.0:1789762734.007201:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762734.033567:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257337375879 time 257337467329 state 0x0 ref 1 00000100:00100000:0.0:1789762734.033586:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014894208:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762734.033629:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 76546048, mtime: 0x6aad9cae, atime: 0x0, ctime: 0x6aad9cae, blocks: 0x2480b 00010000:00010000:0.0:1789762734.033637:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c23 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.033660:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=257s v=(0 0) 00010000:00010000:0.0:1789762734.033665:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.033670:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.033682:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762734.045972:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967644 is committed 00002000:00010000:0.0:1789762734.045989:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:0.0:1789762734.045995:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 149515 -> 151563 00010000:00010000:0.0:1789762734.046006:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c23 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.046017:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c23 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.046024:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.046030:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.046034:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c23 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.046042:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c23 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.046069:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c23 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.046075:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c23 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.046156:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014894208:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12573us (38685us total) trans 4294967644 rc 0/0 00000100:00100000:0.0:1789762734.046166:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 104 00000100:00100000:0.0:1789762734.046175:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257337375879 time 257337467329 state 0x2 ref 0 00000100:00100000:2.0:1789762734.048620:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762734.083651:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257387467329 time 257387550972 state 0x0 ref 1 00000100:00100000:0.0:1789762734.083669:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014894336:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762734.083707:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 77594624, mtime: 0x6aad9cae, atime: 0x0, ctime: 0x6aad9cae, blocks: 0x2500b 00010000:00010000:0.0:1789762734.083714:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c2a lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.083727:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=257s v=(0 0) 00010000:00010000:0.0:1789762734.083731:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.083736:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.083747:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762734.096253:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967645 is committed 00002000:00010000:0.0:1789762734.096272:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:0.0:1789762734.096277:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 151563 -> 153611 00010000:00010000:0.0:1789762734.096288:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c2a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.096298:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c2a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.096305:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.096310:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.096313:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c2a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.096325:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c2a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.096345:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c2a lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.096355:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c2a lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.096425:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014894336:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12758us (47804us total) trans 4294967645 rc 0/0 00000100:00100000:0.0:1789762734.096441:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 105 00000100:00100000:0.0:1789762734.096449:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257387467329 time 257387550972 state 0x2 ref 0 00000100:00100000:0.0:1789762734.133833:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257437550972 time 257437728796 state 0x0 ref 1 00000100:00100000:0.0:1789762734.133885:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014894464:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762734.133939:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 78643200, mtime: 0x6aad9cae, atime: 0x0, ctime: 0x6aad9cae, blocks: 0x2580b 00010000:00010000:0.0:1789762734.133950:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c31 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.133968:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=257s v=(0 0) 00010000:00010000:0.0:1789762734.133974:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.133982:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.133999:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762734.157319:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:0.0:1789762734.157327:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 153611 -> 155659 00010000:00010000:0.0:1789762734.157343:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c31 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.157359:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c31 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.157388:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.157397:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.157403:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c31 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.157414:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c31 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.157436:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c31 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.157445:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c31 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.157538:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014894464:12345-192.168.203.48@tcp:4:dd.500 Request processed in 23681us (58685us total) trans 4294967646 rc 0/0 00000100:00100000:0.0:1789762734.157552:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 106 00000100:00100000:0.0:1789762734.157567:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257437550972 time 257437728796 state 0x2 ref 0 00000100:00100000:0.0:1789762734.184021:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257487728796 time 257487916554 state 0x0 ref 1 00000100:00100000:0.0:1789762734.184048:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014894592:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762734.184149:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 79691776, mtime: 0x6aad9cae, atime: 0x0, ctime: 0x6aad9cae, blocks: 0x2600b 00010000:00010000:0.0:1789762734.184161:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c38 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.184180:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=257s v=(0 0) 00010000:00010000:0.0:1789762734.184187:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.184194:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.184211:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762734.208148:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:0.0:1789762734.208155:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 155659 -> 157707 00010000:00010000:0.0:1789762734.208176:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c38 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.208196:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c38 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.208213:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.208226:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.208235:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.208267:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c38 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.208294:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.208308:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e26b7de2/0x416193e012b02c38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.208426:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014894592:12345-192.168.203.48@tcp:4:dd.500 Request processed in 24379us (46898us total) trans 4294967647 rc 0/0 00000100:00100000:0.0:1789762734.208445:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 107 00000100:00100000:0.0:1789762734.208463:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257487728796 time 257487916554 state 0x2 ref 0 00000100:00100000:2.0:1789762734.211805:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762734.211904:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014894720 00000100:00100000:2.0:1789762734.211915:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014894720 00000100:00100000:2.0:1789762734.211974:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257487728796 time 257487916554 state 0x0 ref 1 00000100:00100000:2.0:1789762734.211991:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257537916554 time 257487916554 state 0x1 ref 1 00000001:00080000:2.0:1789762734.215576:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967434 is committed 00000004:00080000:2.0:1789762734.215591:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 11 committed 00000004:00080000:2.0:1789762734.215606:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 10 committed 00000004:00080000:2.0:1789762734.215645:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 11 committed 00000001:00080000:2.0:1789762734.215653:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967435 is committed 00000004:00080000:2.0:1789762734.215676:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 12 committed 00000001:00080000:2.0:1789762734.215702:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967436 is committed 00000100:00100000:2.0:1789762734.215814:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead6e4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022147840:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762734.215984:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022147712 00000100:00100000:2.0:1789762734.215992:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022147712 00000100:00100000:2.0:1789762734.216074:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022147968 00000100:00100000:2.0:1789762734.216081:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022147968 00000100:00100000:0.0:1789762734.216151:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:0.0:1789762734.216178:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022147712:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789762734.216199:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022147840 00000100:00100000:2.0:1789762734.216210:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022147840 00002000:00080000:0.0:1789762734.216214:0:6649:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:7 count 1 00002000:00010000:0.0:1789762734.216264:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x7:0x0] initial LVB size: 8388608, mtime: 0x6aad9ca9, atime: 0x0, ctime: 0x6aad9ca9, blocks: 0x1 00000100:00100000:2.0:1789762734.216276:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 119 00010000:00010000:0.0:1789762734.216280:0:6649:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e26b7de2/0x416193e012b02c3f lrc: 3/0,1 mode: --/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762734.216303:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022147840:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:0.0:1789762734.216311:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=257s v=(0 0) 00010000:00010000:0.0:1789762734.216321:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x7:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.216333:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e26b7de2/0x416193e012b02c3f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:2.0:1789762734.216339:0:6647:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:9 count 1 00010000:00010000:0.0:1789762734.216377:0:6649:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e26b7de2/0x416193e012b02c3f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.216389:0:6647:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000642c9a96/0x416193e012b02c4d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.216399:0:6649:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e26b7de2/0x416193e012b02c3f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.216420:0:6647:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000593beedc/0x416193e012b028b8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd30d expref: 8 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.216431:0:6649:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e26b7de2/0x416193e012b02c3f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.216453:0:6647:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b028c6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd31b expref: 8 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.216478:0:6647:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b028bf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd314 expref: 8 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.216503:0:6649:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1789762734.216533:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x9:0x0].0x0 since=4s v=(0 0) 00010000:00010000:2.0:1789762734.216541:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762734.216548:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000642c9a96/0x416193e012b02c4d lrc: 6/0,1 mode: --/PW res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.216610:0:6647:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b028bf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd314 expref: 8 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.216658:0:6647:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003c7bf890/0x416193e012b028bf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd314 expref: 8 pid: 10916 timeout: 357 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762734.216703:0:6647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6647:1876702022148096:192.168.203.48@tcp:104: 00010000:00010000:2.0:1789762734.216844:0:6647:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b028c6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd31b expref: 8 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.216909:0:6647:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b028c6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd31b expref: 8 pid: 6647 timeout: 357 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762734.216925:0:6647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4ab5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6647:1876702022148224:192.168.203.48@tcp:104: 00010000:00010000:2.0:1789762734.216980:0:6647:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000593beedc/0x416193e012b028b8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd30d expref: 8 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.216992:0:6647:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000593beedc/0x416193e012b028b8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd30d expref: 8 pid: 6649 timeout: 357 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762734.217022:0:6647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4ab5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6647:1876702022148352:192.168.203.48@tcp:104: 00000100:00100000:2.0:1789762734.217071:0:6647:0:(client.c:2715:ptlrpc_set_wait()) set ffff981eb464a3c0 going to sleep for 11 seconds 00000100:00080000:2.0:1789762734.217130:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762734.217138:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762734.217527:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3274:x1876702022147712:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1358us (1590us total) trans 4294967361 rc 0/0 00000100:00100000:0.0:1789762734.217542:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00010000:00010000:0.0:1789762734.217607:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1789762734.217611:0:3272:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebfc1dc00/2, rc 0, transno 4294967362 00000100:00100000:2.0:1789762734.217624:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc1dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022147968:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1789762734.217985:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762734.217995:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b02c54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.218012:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812d000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762734.218020:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b02c54 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.218054:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b02c54 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.218078:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b02c54 lrc: 1/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.218092:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ce2d89f/0x416193e012b02c54 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.218203:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762734.218221:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762734.218226:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762734.218577:0:6647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfdc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6647:1876702022148096:192.168.203.48@tcp:104: 00000100:00100000:0.0:1789762734.218589:0:6647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4ab5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6647:1876702022148224:192.168.203.48@tcp:104: 00000100:00100000:0.0:1789762734.219232:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762734.219302:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014894976 00000100:00100000:2.0:1789762734.219311:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014894976 00000100:00100000:0.0:1789762734.219341:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762734.219349:0:5791:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b028c6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd31b expref: 9 pid: 6647 timeout: 357 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.219401:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762734.219409:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 52 00000100:00100000:2.0:1789762734.219430:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014894976:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762734.219448:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762734.219463:0:5791:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b028c6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd31b expref: 10 pid: 6647 timeout: 357 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.219528:0:5791:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b028c6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd31b expref: 10 pid: 6647 timeout: 357 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.219578:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098204af1/0x416193e012b028c6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd31b expref: 11 pid: 6647 timeout: 357 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.219604:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762734.219633:0:5792:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 54 00000100:00100000:0.0:1789762734.219649:0:5792:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014895104:12345-192.168.203.48@tcp:103:ldlm_bl.0 00000100:00100000:2.0:1789762734.219657:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014894976:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 231us (423us total) trans 0 rc 0/0 00010000:00010000:0.0:1789762734.219666:0:5792:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762734.219681:0:5792:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000593beedc/0x416193e012b028b8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd30d expref: 10 pid: 6649 timeout: 357 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762734.219683:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 52 00010000:00010000:0.0:1789762734.219726:0:5792:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000593beedc/0x416193e012b028b8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x9:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd30d expref: 9 pid: 6649 timeout: 357 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.219745:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000593beedc/0x416193e012b028b8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd30d expref: 8 pid: 6649 timeout: 357 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.219785:0:5792:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762734.219832:0:5792:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2363:x1876702014895104:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 186us (433us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762734.219844:0:5792:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 54 00010000:00010000:2.0:1789762734.219910:0:6647:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000642c9a96/0x416193e012b02c4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.219926:0:6647:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000642c9a96/0x416193e012b02c4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.219951:0:6647:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000642c9a96/0x416193e012b02c4d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.219962:0:6647:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000642c9a96/0x416193e012b02c4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.220002:0:6647:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762734.220038:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762734.220048:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000642c9a96/0x416193e012b02c4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.220057:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fdfe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.220061:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000642c9a96/0x416193e012b02c4d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.220088:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000642c9a96/0x416193e012b02c4d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.220163:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000642c9a96/0x416193e012b02c4d lrc: 1/0,0 mode: --/PW res: [0x240000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.220182:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000642c9a96/0x416193e012b02c4d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.220229:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762734.220236:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762734.220241:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762734.220771:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3271:x1876702022147840:12345-0@lo:6:osp-syn-0-0.0 Request processed in 4480us (4606us total) trans 4294967649 rc 0/0 00000100:00100000:2.0:1789762734.220784:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 119 00000100:00100000:0.0:1789762734.234115:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257537916554 time 257538015885 state 0x0 ref 1 00000100:00100000:0.0:1789762734.234133:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014894720:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762734.234166:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 80740352, mtime: 0x6aad9cae, atime: 0x0, ctime: 0x6aad9cae, blocks: 0x2680b 00010000:00010000:0.0:1789762734.234174:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02c5b lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.234186:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=257s v=(0 0) 00010000:00010000:0.0:1789762734.234190:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.234195:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02c5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.234206:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02c5b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762734.256149:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967649 is committed 00000100:00100000:2.0:1789762734.260297:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762734.284332:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257588015885 time 257588223044 state 0x0 ref 1 00000100:00100000:0.0:1789762734.284361:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014895232:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762734.284422:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 81788928, mtime: 0x6aad9cae, atime: 0x0, ctime: 0x6aad9cae, blocks: 0x2700b 00010000:00010000:0.0:1789762734.284433:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a592726/0x416193e012b02c62 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.284465:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=257s v=(0 0) 00010000:00010000:0.0:1789762734.284470:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.284477:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004a592726/0x416193e012b02c62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.284493:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004a592726/0x416193e012b02c62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762734.307300:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967651 is committed 00002000:00010000:2.0:1789762734.307371:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:2.0:1789762734.307378:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 159755 -> 161803 00010000:00010000:2.0:1789762734.307395:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004a592726/0x416193e012b02c62 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.307412:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004a592726/0x416193e012b02c62 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.307424:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004a592726/0x416193e012b02c62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.307434:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762734.307452:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004a592726/0x416193e012b02c62 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.307465:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004a592726/0x416193e012b02c62 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.307506:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004a592726/0x416193e012b02c62 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.307516:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004a592726/0x416193e012b02c62 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762734.307634:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014895232:12345-192.168.203.48@tcp:4:dd.500 Request processed in 23278us (47335us total) trans 4294967651 rc 0/0 00000100:00100000:2.0:1789762734.307648:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 109 00000100:00100000:2.0:1789762734.307663:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257588015885 time 257588223044 state 0x2 ref 0 00000100:00100000:2.0:1789762734.311886:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014895360 00000100:00100000:2.0:1789762734.311902:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014895360 00000100:00100000:2.0:1789762734.311974:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257588015885 time 257588223044 state 0x0 ref 1 00000100:00100000:2.0:1789762734.311994:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257638223044 time 257588223044 state 0x1 ref 1 00002000:00010000:2.0:1789762734.357134:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 82837504 -> 83886080 00002000:00010000:2.0:1789762734.357145:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 161803 -> 163851 00010000:00010000:2.0:1789762734.357162:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000005657b01/0x416193e012b02c69 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.357193:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000005657b01/0x416193e012b02c69 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.357206:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000005657b01/0x416193e012b02c69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.357215:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812cc00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762734.357222:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000005657b01/0x416193e012b02c69 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.357233:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000005657b01/0x416193e012b02c69 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.357255:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000005657b01/0x416193e012b02c69 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.357264:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005657b01/0x416193e012b02c69 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762734.357362:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2363:x1876702014895360:12345-192.168.203.48@tcp:4:dd.500 Request processed in 22821us (45581us total) trans 4294967652 rc 0/0 00000100:00100000:2.0:1789762734.357375:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 110 00000100:00100000:2.0:1789762734.357388:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257638223044 time 257638414106 state 0x2 ref 0 00000001:00080000:2.0:1789762734.357443:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967652 is committed 00000100:00100000:2.0:1789762734.384704:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257688414106 time 257688598891 state 0x0 ref 1 00000100:00100000:2.0:1789762734.384732:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014895488:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762734.384784:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 83886080, mtime: 0x6aad9cae, atime: 0x0, ctime: 0x6aad9cae, blocks: 0x2800b 00010000:00010000:2.0:1789762734.384795:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c70 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.384814:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=257s v=(0 0) 00010000:00010000:2.0:1789762734.384821:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762734.384828:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.384845:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762734.407125:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 83886080 -> 84934656 00002000:00010000:2.0:1789762734.407133:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 163851 -> 165899 00010000:00010000:2.0:1789762734.407151:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c70 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.407167:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c70 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.407182:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.407218:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fdea00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762734.407228:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c70 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.407265:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c70 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.407298:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c70 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.407310:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c70 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762734.407431:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014895488:12345-192.168.203.48@tcp:4:dd.500 Request processed in 22702us (46411us total) trans 4294967653 rc 0/0 00000100:00100000:2.0:1789762734.407444:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 111 00000100:00100000:2.0:1789762734.407458:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257688414106 time 257688598891 state 0x2 ref 0 00000100:00100000:2.0:1789762734.411413:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014895616 00000100:00100000:2.0:1789762734.411436:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014895616 00000100:00100000:2.0:1789762734.411515:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257688414106 time 257688598891 state 0x0 ref 1 00000100:00100000:2.0:1789762734.411537:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257738598891 time 257688598891 state 0x1 ref 1 00000100:00100000:2.0:1789762734.434805:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257738598891 time 257738684941 state 0x0 ref 1 00000100:00100000:2.0:1789762734.434879:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014895616:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762734.434940:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 84934656, mtime: 0x6aad9cae, atime: 0x0, ctime: 0x6aad9cae, blocks: 0x2880b 00010000:00010000:2.0:1789762734.434952:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c77 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.434973:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=257s v=(0 0) 00010000:00010000:2.0:1789762734.434979:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762734.434987:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.435006:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007cfaf1d1/0x416193e012b02c77 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762734.484928:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257788684941 time 257788825173 state 0x0 ref 1 00000100:00100000:2.0:1789762734.484953:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014895744:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762734.485003:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 85983232, mtime: 0x6aad9cae, atime: 0x0, ctime: 0x6aad9cae, blocks: 0x2900d 00010000:00010000:2.0:1789762734.485014:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02c7e lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.485032:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=257s v=(0 0) 00010000:00010000:2.0:1789762734.485038:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762734.485045:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02c7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762734.485082:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000c2a8d78/0x416193e012b02c7e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762734.505533:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967655 is committed 00000100:00100000:2.0:1789762734.510396:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014895872 00000100:00100000:2.0:1789762734.510409:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014895872 00000100:00100000:2.0:1789762734.510475:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257788684941 time 257788825173 state 0x0 ref 1 00000100:00100000:2.0:1789762734.510501:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257838825173 time 257788825173 state 0x1 ref 1 00000001:00080000:0.0:1789762734.555808:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967656 is committed 00000001:00080000:2.0:1789762734.609532:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967657 is committed 00000100:00100000:2.0:1789762734.614631:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762734.657029:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967658 is committed 00000100:00100000:2.0:1789762734.661439:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762734.707465:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967659 is committed 00002000:00010000:0.0:1789762734.707477:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 90177536 -> 91226112 00002000:00010000:0.0:1789762734.707487:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 176141 -> 178189 00010000:00010000:0.0:1789762734.707505:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005bcb7a6e/0x416193e012b02c9a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.707522:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005bcb7a6e/0x416193e012b02c9a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.707535:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005bcb7a6e/0x416193e012b02c9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.707544:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfec9200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.707550:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005bcb7a6e/0x416193e012b02c9a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.707562:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005bcb7a6e/0x416193e012b02c9a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.707608:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005bcb7a6e/0x416193e012b02c9a lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.707625:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005bcb7a6e/0x416193e012b02c9a lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.707756:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014896256:12345-192.168.203.48@tcp:4:dd.500 Request processed in 22377us (46314us total) trans 4294967659 rc 0/0 00000100:00100000:0.0:1789762734.707774:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 117 00000100:00100000:0.0:1789762734.707794:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 257989153755 time 257989249093 state 0x2 ref 0 00000100:00100000:0.0:1789762734.711826:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762734.732972:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967361 is committed 00000001:00080000:2.0:1789762734.732993:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967355 is committed 00000001:00080000:2.0:1789762734.733037:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967362 is committed 00000001:00080000:2.0:1789762734.733043:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967356 is committed 00000001:00080000:2.0:1789762734.733055:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967360 is committed 00002000:00010000:0.0:1789762734.758156:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 91226112 -> 92274688 00002000:00010000:0.0:1789762734.758166:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 178189 -> 180237 00010000:00010000:0.0:1789762734.758182:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000adddab01/0x416193e012b02ca1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.758199:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000adddab01/0x416193e012b02ca1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.758211:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000adddab01/0x416193e012b02ca1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.758220:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfec8a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.758226:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000adddab01/0x416193e012b02ca1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.758250:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000adddab01/0x416193e012b02ca1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.758275:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000adddab01/0x416193e012b02ca1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.758290:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000adddab01/0x416193e012b02ca1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.758403:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014896384:12345-192.168.203.48@tcp:4:dd.500 Request processed in 22866us (46587us total) trans 4294967660 rc 0/0 00000100:00100000:0.0:1789762734.758419:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 118 00000100:00100000:0.0:1789762734.758433:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258039249093 time 258039405366 state 0x2 ref 0 00000100:00100000:0.0:1789762734.785660:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258089405366 time 258089541394 state 0x0 ref 1 00000100:00100000:0.0:1789762734.785684:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014896512:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762734.785732:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 92274688, mtime: 0x6aad9cae, atime: 0x0, ctime: 0x6aad9cae, blocks: 0x2c00d 00010000:00010000:0.0:1789762734.785742:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ca8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.785760:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762734.785767:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.785777:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.785813:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762734.810399:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:0.0:1789762734.810408:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 180237 -> 182285 00010000:00010000:0.0:1789762734.810425:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ca8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.810442:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ca8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.810453:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.810461:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.810468:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ca8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.810480:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ca8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.810501:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ca8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.810510:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ca8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.810634:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014896512:12345-192.168.203.48@tcp:4:dd.500 Request processed in 24951us (48047us total) trans 4294967661 rc 0/0 00000100:00100000:0.0:1789762734.810651:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 119 00000100:00100000:0.0:1789762734.810666:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258089405366 time 258089541394 state 0x2 ref 0 00000100:00100000:0.0:1789762734.835783:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258139541394 time 258139672648 state 0x0 ref 1 00000100:00100000:0.0:1789762734.835812:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014896640:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762734.835901:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 93323264, mtime: 0x6aad9cae, atime: 0x0, ctime: 0x6aad9cae, blocks: 0x2c80d 00010000:00010000:0.0:1789762734.835913:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02caf lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.835934:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762734.835942:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.835954:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02caf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.835980:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02caf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762734.857075:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:0.0:1789762734.857110:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb mtime from disk: 1789762734 -> 1789762735 00002000:00010000:0.0:1789762734.857115:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb ctime from disk: 1789762734 -> 1789762735 00002000:00010000:0.0:1789762734.857118:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 182285 -> 184333 00010000:00010000:0.0:1789762734.857148:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02caf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.857165:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02caf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.857176:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02caf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.857185:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.857191:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02caf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.857203:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02caf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.857224:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02caf lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.857233:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02caf lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.857331:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014896640:12345-192.168.203.48@tcp:4:dd.500 Request processed in 21523us (42599us total) trans 4294967662 rc 0/0 00000100:00100000:0.0:1789762734.857347:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 120 00000100:00100000:0.0:1789762734.857361:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258139541394 time 258139672648 state 0x2 ref 0 00000100:00100000:0.0:1789762734.885903:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258189672648 time 258189799952 state 0x0 ref 1 00000100:00100000:0.0:1789762734.885928:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014896768:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762734.885989:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 94371840, mtime: 0x6aad9caf, atime: 0x0, ctime: 0x6aad9caf, blocks: 0x2d00d 00010000:00010000:0.0:1789762734.886004:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cb6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.886030:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762734.886039:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.886048:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.886065:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762734.906701:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 94371840 -> 95420416 00002000:00010000:0.0:1789762734.906710:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 184333 -> 186381 00010000:00010000:0.0:1789762734.906724:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cb6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.906740:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cb6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.906751:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.906779:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.906785:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.906796:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.906817:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.906826:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.906952:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014896768:12345-192.168.203.48@tcp:4:dd.500 Request processed in 21023us (45545us total) trans 4294967663 rc 0/0 00000100:00100000:0.0:1789762734.906970:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 121 00000100:00100000:0.0:1789762734.906990:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258189672648 time 258189799952 state 0x2 ref 0 00000100:00100000:0.0:1789762734.936038:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258239799952 time 258239936717 state 0x0 ref 1 00000100:00100000:0.0:1789762734.936062:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014896896:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762734.936149:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 95420416, mtime: 0x6aad9caf, atime: 0x0, ctime: 0x6aad9caf, blocks: 0x2d80d 00010000:00010000:0.0:1789762734.936161:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cbd lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.936190:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762734.936196:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.936203:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.936219:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762734.956993:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 95420416 -> 96468992 00002000:00010000:0.0:1789762734.957005:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 186381 -> 188429 00010000:00010000:0.0:1789762734.957023:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cbd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.957046:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cbd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.957062:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.957072:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762734.957078:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cbd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.957124:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cbd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.957165:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cbd lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.957175:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cbd lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762734.957296:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014896896:12345-192.168.203.48@tcp:4:dd.500 Request processed in 21235us (46618us total) trans 4294967664 rc 0/0 00000100:00100000:0.0:1789762734.957311:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 122 00000100:00100000:0.0:1789762734.957324:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258239799952 time 258239936717 state 0x2 ref 0 00000100:00100000:0.0:1789762734.986160:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258289936717 time 258290056459 state 0x0 ref 1 00000100:00100000:0.0:1789762734.986204:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014897024:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762734.986259:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 96468992, mtime: 0x6aad9caf, atime: 0x0, ctime: 0x6aad9caf, blocks: 0x2e00d 00010000:00010000:0.0:1789762734.986270:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cc4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.986290:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762734.986312:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762734.986320:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762734.986337:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762735.009994:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 96468992 -> 97517568 00002000:00010000:0.0:1789762735.010007:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 188429 -> 190477 00010000:00010000:0.0:1789762735.010031:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cc4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.010056:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cc4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.010074:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.010085:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762735.010092:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cc4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.010136:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cc4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.010163:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cc4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.010173:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cc4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762735.010293:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014897024:12345-192.168.203.48@tcp:4:dd.500 Request processed in 24108us (49342us total) trans 4294967665 rc 0/0 00000100:00100000:0.0:1789762735.010325:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 123 00000100:00100000:0.0:1789762735.010358:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258289936717 time 258290056459 state 0x2 ref 0 00000100:00080000:2.0:1789762735.014707:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762735.036286:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258340056459 time 258340182459 state 0x0 ref 1 00000100:00100000:0.0:1789762735.036313:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014897152:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762735.036373:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 97517568, mtime: 0x6aad9caf, atime: 0x0, ctime: 0x6aad9caf, blocks: 0x2e80d 00010000:00010000:0.0:1789762735.036388:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ccb lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.036414:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762735.036422:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762735.036434:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.036459:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762735.060999:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 97517568 -> 98566144 00002000:00010000:0.0:1789762735.061011:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 190477 -> 192525 00010000:00010000:0.0:1789762735.061034:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ccb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.061057:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ccb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.061126:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ccb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.061137:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762735.061143:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ccb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.061155:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ccb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.061179:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ccb lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.061189:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ccb lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762735.061320:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014897152:12345-192.168.203.48@tcp:4:dd.500 Request processed in 25011us (46860us total) trans 4294967666 rc 0/0 00000100:00100000:0.0:1789762735.061334:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 124 00000100:00100000:0.0:1789762735.061351:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258340056459 time 258340182459 state 0x2 ref 0 00000100:00100000:0.0:1789762735.065808:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762735.086502:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258390182459 time 258390352710 state 0x0 ref 1 00000100:00100000:0.0:1789762735.086534:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014897280:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762735.086615:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 98566144, mtime: 0x6aad9caf, atime: 0x0, ctime: 0x6aad9caf, blocks: 0x2f00d 00010000:00010000:0.0:1789762735.086626:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.086647:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762735.086653:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762735.086661:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.086679:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762735.109493:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 98566144 -> 99614720 00002000:00010000:0.0:1789762735.109502:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 192525 -> 194573 00010000:00010000:0.0:1789762735.109521:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.109538:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.109548:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.109556:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762735.109562:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.109588:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.109609:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.109617:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762735.109737:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014897280:12345-192.168.203.48@tcp:4:dd.500 Request processed in 23188us (43906us total) trans 4294967667 rc 0/0 00000100:00100000:0.0:1789762735.109758:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 125 00000100:00100000:0.0:1789762735.109779:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258390182459 time 258390352710 state 0x2 ref 0 00000100:00100000:0.0:1789762735.113760:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762735.136534:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258440352710 time 258440436822 state 0x0 ref 1 00000100:00100000:0.0:1789762735.136550:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014897408:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762735.136588:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 99614720, mtime: 0x6aad9caf, atime: 0x0, ctime: 0x6aad9caf, blocks: 0x2f80d 00010000:00010000:0.0:1789762735.136595:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.136609:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762735.136612:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762735.136626:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.136638:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762735.152705:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 99614720 -> 100663296 00002000:00010000:0.0:1789762735.152711:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 194573 -> 196621 00010000:00010000:0.0:1789762735.152721:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.152732:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.152739:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.152745:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762735.152749:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.152756:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.152771:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.152788:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cd9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762735.152885:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014897408:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16301us (39092us total) trans 4294967668 rc 0/0 00000100:00100000:0.0:1789762735.152898:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 126 00000100:00100000:0.0:1789762735.152911:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258440352710 time 258440436822 state 0x2 ref 0 00000100:00100000:0.0:1789762735.186621:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258490436822 time 258490521001 state 0x0 ref 1 00000100:00100000:0.0:1789762735.186641:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014897536:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762735.186685:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 100663296, mtime: 0x6aad9caf, atime: 0x0, ctime: 0x6aad9caf, blocks: 0x3000d 00010000:00010000:0.0:1789762735.186692:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.186706:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762735.186710:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762735.186722:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.186735:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762735.200409:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 100663296 -> 101711872 00002000:00010000:0.0:1789762735.200420:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 196621 -> 198669 00010000:00010000:0.0:1789762735.200433:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.200443:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.200450:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.200455:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762735.200459:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.200466:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.200481:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.200489:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762735.200575:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014897536:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13937us (44704us total) trans 4294967669 rc 0/0 00000100:00100000:0.0:1789762735.200585:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 127 00000100:00100000:0.0:1789762735.200595:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258490436822 time 258490521001 state 0x2 ref 0 00000100:00100000:2.0:1789762735.203316:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762735.236766:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258540521001 time 258540663057 state 0x0 ref 1 00000100:00100000:0.0:1789762735.236788:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81380e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014897664:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762735.236835:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 101711872, mtime: 0x6aad9caf, atime: 0x0, ctime: 0x6aad9caf, blocks: 0x3080d 00010000:00010000:0.0:1789762735.236843:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.236872:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762735.236876:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762735.236882:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.236899:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762735.252371:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:0.0:1789762735.252379:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 198669 -> 200717 00010000:00010000:0.0:1789762735.252396:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.252427:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.252439:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.252463:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762735.252488:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.252508:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.252540:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.252558:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02ce7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762735.252683:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81380e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014897664:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15898us (49389us total) trans 4294967670 rc 0/0 00000100:00100000:0.0:1789762735.252696:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 128 00000100:00100000:0.0:1789762735.252710:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258540521001 time 258540663057 state 0x2 ref 0 00000100:00100000:2.0:1789762735.255807:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014897792 00000100:00100000:2.0:1789762735.255816:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014897792 00000100:00100000:2.0:1789762735.255872:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258540521001 time 258540663057 state 0x0 ref 1 00000100:00100000:2.0:1789762735.255885:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258590663057 time 258540663057 state 0x1 ref 1 00000100:00100000:0.0:1789762735.286881:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258590663057 time 258590781805 state 0x0 ref 1 00000100:00100000:0.0:1789762735.286900:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014897792:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762735.286942:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 102760448, mtime: 0x6aad9caf, atime: 0x0, ctime: 0x6aad9caf, blocks: 0x3100d 00010000:00010000:0.0:1789762735.286953:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cee lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.286977:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762735.286982:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762735.286989:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.287000:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762735.300896:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:0.0:1789762735.300902:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 200717 -> 202765 00010000:00010000:0.0:1789762735.300915:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.300926:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.300933:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.300939:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762735.300943:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.300951:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.300980:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cee lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.300987:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cee lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762735.301066:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014897792:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14166us (45337us total) trans 4294967671 rc 0/0 00000100:00100000:0.0:1789762735.301078:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 129 00000100:00100000:0.0:1789762735.301124:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258590663057 time 258590781805 state 0x2 ref 0 00000100:00100000:0.0:1789762735.336945:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258640781805 time 258640846451 state 0x0 ref 1 00000100:00100000:0.0:1789762735.336964:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014897920:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762735.337005:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 103809024, mtime: 0x6aad9caf, atime: 0x0, ctime: 0x6aad9caf, blocks: 0x3180d 00010000:00010000:0.0:1789762735.337012:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cf5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.337026:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=258s v=(0 0) 00010000:00010000:0.0:1789762735.337030:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762735.337034:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.337045:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762735.348819:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb size from disk: 103809024 -> 104857600 00002000:00010000:0.0:1789762735.348826:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xb:0x0] updating lvb blocks from disk: 202765 -> 204813 00010000:00010000:0.0:1789762735.348839:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cf5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.348866:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cf5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.348875:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.348883:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762735.348888:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cf5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.348898:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cf5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.348916:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cf5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.348925:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000053aa61e5/0x416193e012b02cf5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762735.348995:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014897920:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12033us (45373us total) trans 4294967672 rc 0/0 00000100:00100000:0.0:1789762735.349012:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 130 00000100:00100000:0.0:1789762735.349023:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258640781805 time 258640846451 state 0x2 ref 0 00000100:00100000:2.0:1789762735.350521:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762735.350615:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762735.391569:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762735.391614:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014898432 00000100:00100000:0.0:1789762735.391624:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014898432 00000100:00100000:0.0:1789762735.391676:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 180 00000100:00100000:0.0:1789762735.391685:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:12555:x1876702014898432:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762735.391706:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762735.391722:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2c00/0x416193e012b02cfc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd36f expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.391807:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2e00/0x416193e012b02d03 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.391822:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2e00/0x416193e012b02d03 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x1d:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.391835:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2e00/0x416193e012b02d03 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x1d:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762735.391859:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1d:0x0]: returning size 104857600 00010000:00010000:0.0:1789762735.391964:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2c00/0x416193e012b02cfc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x1b:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd36f expref: 13 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.391978:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2e00/0x416193e012b02d03 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x1d:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd36f expref: 12 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.391984:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1d:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:0.0:1789762735.391999:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:0.0:1789762735.392002:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebf4f2e00, rc 0) 00000100:00100000:0.0:1789762735.392050:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81380000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:12555:x1876702014898432:12345-192.168.203.48@tcp:101:dd.500 Request processed in 367us (489us total) trans 4294967439 rc 0/0 00000100:00100000:0.0:1789762735.392056:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 180 00000100:00100000:2.0:1789762735.395910:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762735.401171:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762735.405232:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762735.405289:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014898944 00000100:00100000:2.0:1789762735.405295:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014898944 00000100:00100000:2.0:1789762735.405334:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 1 deadline 258799951119 time 258705199789 state 0x0 ref 1 00000100:00100000:2.0:1789762735.405348:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 1 deadline 258805199789 time 258705199789 state 0x1 ref 1 00000100:00100000:2.0:1789762735.405372:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 258805199789 time 258709281274 state 0x0 ref 1 00000100:00100000:2.0:1789762735.405382:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014898944:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762735.405397:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d26 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.405406:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762735.405409:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762735.405413:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d26 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.405421:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d26 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.407525:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d26 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.407542:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d26 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.407548:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d26 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.407553:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bd400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762735.407555:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d26 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.407560:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d26 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.407570:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d26 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.407575:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d26 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762735.407620:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014898944:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2237us (2386us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762735.407629:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 133 00000100:00100000:2.0:1789762735.407639:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 258805199789 time 258709281274 state 0x2 ref 0 00000100:00100000:2.0:1789762735.409201:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762735.510486:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762735.605548:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 258909361396 time 258909445051 state 0x0 ref 1 00000100:00100000:2.0:1789762735.605573:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81383480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014899200:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762735.605609:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d34 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.605622:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762735.605626:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762735.605630:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d34 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762735.605641:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d34 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.608697:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d34 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.608708:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d34 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.608716:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d34 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.608722:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e857bd400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762735.608725:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d34 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.608731:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d34 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.608744:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d34 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.608749:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000034f4b48b/0x416193e012b02d34 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762735.608823:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81383480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014899200:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3259us (98338us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762735.608832:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 135 00000100:00100000:0.0:1789762735.608854:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 258909361396 time 258909445051 state 0x2 ref 0 00000100:00100000:0.0:1789762735.611024:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762735.710897:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762735.805933:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 259109671773 time 259109819546 state 0x0 ref 1 00000100:00100000:0.0:1789762735.805967:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81380700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014899456:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762735.806020:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d42 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.806043:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762735.806048:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762735.806056:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d42 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.806075:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d42 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.811797:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d42 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.811816:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d42 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.811868:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d42 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.811880:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762735.811887:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d42 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.811900:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d42 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.811923:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d42 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762735.811935:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d42 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762735.812015:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81380700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014899456:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6058us (101115us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762735.812027:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 137 00000100:00100000:0.0:1789762735.812040:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 259109671773 time 259109819546 state 0x2 ref 0 00000100:00100000:0.0:1789762735.815581:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762736.006299:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 259309996894 time 259310179539 state 0x0 ref 1 00000100:00100000:0.0:1789762736.006347:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014899712:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762736.006439:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d50 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.006462:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762736.006468:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762736.006476:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d50 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.006494:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d50 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.011816:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d50 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.011860:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d50 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.011893:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d50 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.011910:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762736.011917:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d50 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.011930:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d50 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.011971:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d50 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.011981:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d50 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762736.012067:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014899712:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5737us (96346us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762736.012079:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 139 00000100:00100000:0.0:1789762736.012138:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 259309996894 time 259310179539 state 0x2 ref 0 00010000:00010000:0.0:1789762736.055193:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762736.206633:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 259510368724 time 259510513489 state 0x0 ref 1 00000100:00100000:0.0:1789762736.206676:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014899968:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762736.206738:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d5e lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.206792:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762736.206798:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762736.206808:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d5e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.206828:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d5e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.211350:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d5e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.211368:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d5e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.211379:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d5e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.211388:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762736.211393:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d5e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.211401:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d5e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.211423:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d5e lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.211431:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02d5e lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762736.211504:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014899968:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4843us (97244us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762736.211521:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 141 00000100:00100000:0.0:1789762736.211547:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 259510368724 time 259510513489 state 0x2 ref 0 00000100:00100000:2.0:1789762736.226704:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762736.226749:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762736.226781:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014900224 00000100:00100000:0.0:1789762736.226795:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014900224 00000100:00100000:2.0:1789762736.226807:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762736.226907:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 274 00000100:00100000:2.0:1789762736.226914:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762736.226935:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702014900224:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762736.226984:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 173 00000100:00100000:2.0:1789762736.227010:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81383b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702014900352:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762736.227031:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702014900224:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 103us (332us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762736.227050:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 274 00000100:00100000:2.0:1789762736.227147:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81383b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702014900352:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 143us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762736.227165:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 173 00000100:00080000:0.0:1789762736.227200:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762736.227212:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762736.247570:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022148480 00000100:00100000:2.0:1789762736.247594:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022148480 00000100:00080000:0.0:1789762736.247668:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762736.247677:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 75 00000100:00100000:2.0:1789762736.247695:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022148480:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762736.247740:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3768582144 avail 3752853504 00000100:00100000:2.0:1789762736.247786:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022148480:12345-0@lo:13:osp-pre-0-0.0 Request processed in 99us (305us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762736.247793:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 75 00000100:00100000:2.0:1789762736.316519:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762736.406985:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 259710701934 time 259710872380 state 0x0 ref 1 00000100:00100000:0.0:1789762736.407021:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014900736:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762736.407073:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d6c lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.407152:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762736.407161:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762736.407174:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d6c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.407203:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d6c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.412508:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d6c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.412529:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d6c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.412548:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d6c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.412565:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe59200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762736.412575:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d6c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.412596:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d6c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.412628:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d6c lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.412639:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d6c lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762736.412720:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81381180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014900736:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5711us (96209us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762736.412733:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 143 00000100:00100000:0.0:1789762736.412760:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 259710701934 time 259710872380 state 0x2 ref 0 00000100:00100000:0.0:1789762736.416136:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762736.517003:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762736.607337:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 259911067091 time 259911225053 state 0x0 ref 1 00000100:00100000:0.0:1789762736.607369:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014900992:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762736.607417:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d7a lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.607437:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762736.607442:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762736.607450:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d7a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.607479:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d7a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.612810:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d7a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.612828:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d7a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.612867:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d7a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.612895:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe59200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762736.612901:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d7a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.612913:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d7a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.612936:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d7a lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.612946:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d7a lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762736.613023:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014900992:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5663us (96022us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762736.613034:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 145 00000100:00100000:0.0:1789762736.613046:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 259911067091 time 259911225053 state 0x2 ref 0 00000100:00100000:0.0:1789762736.616453:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762736.716900:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762736.759365:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022148992:0@lo:400:kworker.0 00000100:00100000:2.0:1789762736.759435:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022148736:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.759533:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762736.759614:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762736.759666:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022148992 00000100:00100000:0.0:1789762736.759677:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022148992 00000100:00100000:2.0:1789762736.759710:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdb9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022149248:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.759767:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:2.0:1789762736.759811:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762736.759813:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022148992:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762736.759892:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762736.759910:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebf512d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022149376:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.759961:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022148992:12345-0@lo:400:kworker.0 Request processed in 173us (441us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762736.759975:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:2.0:1789762736.760004:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762736.760016:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022149248 00000100:00100000:0.0:1789762736.760025:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022149248 00000100:00100000:2.0:1789762736.760138:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022148736:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.760151:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:0.0:1789762736.760173:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf513800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022149248:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762736.760211:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022149120 00000100:00100000:2.0:1789762736.760220:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022149120 00000100:00100000:0.0:1789762736.760256:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf513800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022149248:12345-0@lo:400:kworker.0 Request processed in 88us (460us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762736.760272:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00100000:2.0:1789762736.760295:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 85 00000100:00100000:0.0:1789762736.760306:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022148864 00000100:00100000:0.0:1789762736.760316:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022148864 00000100:00100000:2.0:1789762736.760324:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022149120:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.760401:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:2.0:1789762736.760411:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022149120:12345-0@lo:400:kworker.0 Request processed in 93us (707us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762736.760431:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022148864:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762736.760446:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 85 00000100:00100000:2.0:1789762736.760533:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdb9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022149248:0@lo:400:kworker.0 00000100:00100000:2.0:1789762736.760583:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebf512d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022149376:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.760597:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022148864:12345-0@lo:400:kworker.0 Request processed in 169us (356us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762736.760624:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:2.0:1789762736.760737:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdb8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022148864:0@lo:400:kworker.0 00000100:00100000:0.0:1789762736.807727:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 260111428631 time 260111612280 state 0x0 ref 1 00000100:00100000:0.0:1789762736.807766:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81383100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014901248:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762736.807820:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d88 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.807872:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762736.807879:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762736.807887:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d88 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.807927:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d88 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.814156:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d88 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.814177:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d88 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.814191:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d88 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.814202:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe59200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762736.814210:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d88 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.814237:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d88 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.814265:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d88 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762736.814277:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d88 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762736.814408:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81383100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014901248:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6652us (97505us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762736.814424:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 147 00000100:00100000:0.0:1789762736.814440:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 260111428631 time 260111612280 state 0x2 ref 0 00000100:00100000:0.0:1789762736.817930:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762737.008041:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 260311814868 time 260311927977 state 0x0 ref 1 00000100:00100000:0.0:1789762737.008132:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014901504:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762737.008186:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d96 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.008209:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762737.008215:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762737.008224:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d96 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.008243:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d96 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.013458:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d96 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.013477:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d96 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.013509:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d96 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.013520:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe59200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762737.013527:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d96 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.013538:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d96 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.013560:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d96 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.013570:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02d96 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762737.013662:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014901504:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5594us (96405us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762737.013680:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 149 00000100:00100000:0.0:1789762737.013698:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 260311814868 time 260311927977 state 0x2 ref 0 00010000:00010000:2.0:1789762737.079218:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762737.113049:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762737.208333:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 260512039027 time 260512215557 state 0x0 ref 1 00000100:00100000:0.0:1789762737.208378:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf512d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014901760:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762737.208459:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02da4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.208481:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762737.208487:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762737.208495:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02da4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.208512:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02da4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.212682:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02da4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.212702:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02da4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.212715:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02da4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.212725:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe59200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762737.212732:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02da4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.212743:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02da4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.212766:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02da4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.212777:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02da4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762737.212877:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf512d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014901760:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4514us (99824us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762737.212903:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 151 00000100:00100000:2.0:1789762737.212917:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 260512039027 time 260512215557 state 0x2 ref 0 00000100:00100000:2.0:1789762737.216259:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762737.408624:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 260712333506 time 260712512333 state 0x0 ref 1 00000100:00100000:2.0:1789762737.408659:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf564a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014902016:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762737.408710:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02db2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.408731:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762737.408737:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762737.408745:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02db2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.408762:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02db2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.413895:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02db2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.413908:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02db2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.413935:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02db2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.413968:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812ce00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762737.413976:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02db2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.413987:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02db2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.414007:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02db2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.414016:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02db2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762737.414143:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf564a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014902016:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5496us (97765us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762737.414154:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 153 00000100:00100000:2.0:1789762737.414167:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 260712333506 time 260712512333 state 0x2 ref 0 00000100:00100000:0.0:1789762737.417304:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762737.516691:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762737.608857:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 260912629472 time 260912751798 state 0x0 ref 1 00000100:00100000:2.0:1789762737.608885:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014902272:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762737.608922:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dc0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.608953:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762737.608957:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762737.608962:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dc0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.608974:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dc0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.613245:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dc0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.613266:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dc0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.613275:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dc0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.613282:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812ce00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762737.613287:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dc0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.613294:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dc0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.613310:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dc0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.613330:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dc0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762737.613400:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014902272:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4522us (96707us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762737.613413:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 155 00000100:00100000:2.0:1789762737.613427:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 260912629472 time 260912751798 state 0x2 ref 0 00000100:00100000:0.0:1789762737.615651:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762737.809075:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261112831848 time 261112967249 state 0x0 ref 1 00000100:00100000:2.0:1789762737.809177:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf564000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014902528:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762737.809249:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dce lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.809272:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762737.809278:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762737.809287:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dce lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762737.809306:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dce lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.813661:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dce lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.813676:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dce lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.813687:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dce lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.813695:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812ce00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762737.813701:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dce lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.813729:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dce lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.813747:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dce lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762737.813754:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001cfb4ff1/0x416193e012b02dce lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762737.813839:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf564000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014902528:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4666us (98505us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762737.813851:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 157 00000100:00100000:0.0:1789762737.813863:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261112831848 time 261112967249 state 0x2 ref 0 00000100:00100000:2.0:1789762737.816273:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762738.009315:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261313127532 time 261313210105 state 0x0 ref 1 00000100:00100000:0.0:1789762738.009348:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014902784:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762738.009387:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02ddc lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.009402:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762738.009405:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762738.009410:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02ddc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.009438:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02ddc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.012392:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02ddc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.012406:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02ddc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.012416:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02ddc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.012424:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762738.012429:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02ddc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.012440:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02ddc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.012458:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02ddc lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.012465:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02ddc lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762738.012527:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf567480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014902784:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3196us (97160us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762738.012534:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 159 00000100:00100000:0.0:1789762738.012542:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261313127532 time 261313210105 state 0x2 ref 0 00000100:00100000:2.0:1789762738.014439:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762738.103173:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762738.209558:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261513360745 time 261513452697 state 0x0 ref 1 00000100:00100000:0.0:1789762738.209585:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014903040:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762738.209622:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02dea lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.209636:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762738.209640:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762738.209644:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02dea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.209656:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02dea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.212694:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02dea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.212705:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02dea lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.212713:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02dea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.212719:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f2200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762738.212723:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02dea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.212729:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02dea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.212745:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02dea lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.212751:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038e42c74/0x416193e012b02dea lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762738.212805:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf565c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014903040:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3232us (97201us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762738.212812:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 161 00000100:00100000:2.0:1789762738.212820:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261513360745 time 261513452697 state 0x2 ref 0 00000100:00100000:2.0:1789762738.214875:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014903168 00000100:00100000:2.0:1789762738.214884:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014903168 00000100:00100000:2.0:1789762738.214919:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261513360745 time 261513452697 state 0x0 ref 1 00000100:00100000:2.0:1789762738.214930:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261613452697 time 261513452697 state 0x1 ref 1 00000100:00100000:0.0:1789762738.314872:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762738.409794:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261713570384 time 261713689702 state 0x0 ref 1 00000100:00100000:2.0:1789762738.409832:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014903296:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762738.409870:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b27cd2e0/0x416193e012b02df8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.409884:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762738.409888:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762738.409893:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b27cd2e0/0x416193e012b02df8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762738.409904:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b27cd2e0/0x416193e012b02df8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.412989:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b27cd2e0/0x416193e012b02df8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.413017:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b27cd2e0/0x416193e012b02df8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.413025:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b27cd2e0/0x416193e012b02df8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.413031:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfb7be00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762738.413035:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b27cd2e0/0x416193e012b02df8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.413043:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b27cd2e0/0x416193e012b02df8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.413061:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b27cd2e0/0x416193e012b02df8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.413066:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b27cd2e0/0x416193e012b02df8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762738.413188:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014903296:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3374us (98314us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762738.413198:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 163 00000100:00100000:0.0:1789762738.413209:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261713570384 time 261713689702 state 0x2 ref 0 00000100:00100000:0.0:1789762738.415198:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762738.415287:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014903424 00000100:00100000:2.0:1789762738.415296:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014903424 00000100:00100000:2.0:1789762738.415337:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261713570384 time 261713689702 state 0x0 ref 1 00000100:00100000:2.0:1789762738.415348:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261813689702 time 261713689702 state 0x1 ref 1 00000100:00100000:0.0:1789762738.610009:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261913824042 time 261913900855 state 0x0 ref 1 00000100:00100000:0.0:1789762738.610038:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf567b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014903552:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762738.610103:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e06 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.610122:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762738.610128:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762738.610136:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e06 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.610154:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e06 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.612967:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e06 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.612980:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e06 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.612988:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e06 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.612994:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fde000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762738.613000:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e06 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.613022:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e06 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.613043:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e06 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.613048:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e06 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762738.613179:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf567b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014903552:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3148us (97776us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762738.613192:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 165 00000100:00100000:0.0:1789762738.613203:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261913824042 time 261913900855 state 0x2 ref 0 00000100:00100000:0.0:1789762738.615709:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014903680 00000100:00100000:0.0:1789762738.615719:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014903680 00000100:00100000:0.0:1789762738.615770:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 261913824042 time 261913900855 state 0x0 ref 1 00000100:00100000:0.0:1789762738.615783:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262013900855 time 261913900855 state 0x1 ref 1 00000100:00100000:3.0:1789762738.713772:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 166 00000100:00100000:3.0:1789762738.713791:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262013900855 time 262014042772 state 0x2 ref 0 00000100:00100000:3.0:1789762738.715499:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762738.810262:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262114042772 time 262114160724 state 0x0 ref 1 00000100:00100000:0.0:1789762738.810282:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf566a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014903808:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762738.810316:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e14 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.810338:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762738.810342:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762738.810347:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e14 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.810357:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e14 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.814350:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e14 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.814361:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e14 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.814368:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e14 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.814374:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fde000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762738.814377:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e14 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.814383:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e14 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.814395:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e14 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762738.814412:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002f386aac/0x416193e012b02e14 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762738.814465:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf566a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014903808:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4188us (98965us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762738.814474:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 167 00000100:00100000:0.0:1789762738.814484:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262114042772 time 262114160724 state 0x2 ref 0 00000100:00100000:2.0:1789762738.816961:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762738.817000:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014903936 00000100:00100000:0.0:1789762738.817008:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014903936 00000100:00100000:0.0:1789762738.817043:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262114042772 time 262114160724 state 0x0 ref 1 00000100:00100000:0.0:1789762738.817053:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262214160724 time 262114160724 state 0x1 ref 1 00000100:00100000:3.0:1789762738.910339:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262214160724 time 262214236349 state 0x0 ref 1 00000100:00100000:3.0:1789762738.910362:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf510000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014903936:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762738.910396:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e1b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762738.910409:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762738.910413:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762738.910418:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e1b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762738.910429:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e1b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762738.913435:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e1b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762738.913444:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e1b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762738.913451:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e1b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762738.913470:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfb7b800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762738.913475:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e1b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762738.913482:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e1b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762738.913494:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e1b lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762738.913499:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e1b lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762738.913545:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf510000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014903936:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3190us (96584us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762738.913555:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 168 00000100:00100000:3.0:1789762738.913565:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262214160724 time 262214236349 state 0x2 ref 0 00000100:00100000:0.0:1789762738.916226:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014904064 00000100:00100000:0.0:1789762738.916235:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014904064 00000100:00100000:0.0:1789762738.916270:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262214160724 time 262214236349 state 0x0 ref 1 00000100:00100000:0.0:1789762738.916281:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262314236349 time 262214236349 state 0x1 ref 1 00000100:00100000:0.0:1789762739.010403:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262314236349 time 262314300011 state 0x0 ref 1 00000100:00100000:0.0:1789762739.010436:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014904064:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762739.010487:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02e22 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.010501:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762739.010504:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762739.010509:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02e22 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.010520:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02e22 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.013233:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02e22 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.013245:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02e22 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.013252:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02e22 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.013258:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aac00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762739.013261:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02e22 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.013267:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02e22 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.013282:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02e22 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.013288:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02e22 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762739.013335:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014904064:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2919us (97156us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762739.013342:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 169 00000100:00100000:2.0:1789762739.013356:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262314236349 time 262314300011 state 0x2 ref 0 00000100:00100000:0.0:1789762739.015113:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014904192 00000100:00100000:0.0:1789762739.015121:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014904192 00000100:00100000:0.0:1789762739.015157:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262314236349 time 262314300011 state 0x0 ref 1 00000100:00100000:0.0:1789762739.015167:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262414300011 time 262314300011 state 0x1 ref 1 00000100:00100000:3.0:1789762739.110514:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262414300011 time 262414408989 state 0x0 ref 1 00000100:00100000:3.0:1789762739.110533:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014904192:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762739.110559:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e29 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.110586:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762739.110589:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762739.110594:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e29 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.110604:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e29 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.113714:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e29 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.113723:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e29 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.113730:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e29 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.113736:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfb7b800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762739.113740:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e29 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.113746:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e29 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.113758:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e29 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.113781:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e29 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762739.113835:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014904192:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3307us (98774us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762739.113842:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 170 00000100:00100000:3.0:1789762739.113849:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262414300011 time 262414408989 state 0x2 ref 0 00000100:00100000:0.0:1789762739.115554:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:3.0:1789762739.127221:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762739.210618:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262514408989 time 262514514166 state 0x0 ref 1 00000100:00100000:2.0:1789762739.210641:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014904320:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762739.210676:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b02e30 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.210690:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762739.210694:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762739.210698:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b02e30 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.210709:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b02e30 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.213859:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b02e30 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.213872:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b02e30 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.213882:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b02e30 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.213904:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762739.213910:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b02e30 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.213920:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b02e30 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.213939:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b02e30 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.213945:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b02e30 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762739.213992:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014904320:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3360us (98439us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762739.213999:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 171 00000100:00100000:0.0:1789762739.214008:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262514408989 time 262514514166 state 0x2 ref 0 00000100:00100000:0.0:1789762739.215962:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762739.216059:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014904448 00000100:00100000:2.0:1789762739.216071:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014904448 00000100:00100000:2.0:1789762739.216173:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262514408989 time 262514514166 state 0x0 ref 1 00000100:00100000:2.0:1789762739.216196:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262614514166 time 262514514166 state 0x1 ref 1 00000100:00100000:3.0:1789762739.310730:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262614514166 time 262614625214 state 0x0 ref 1 00000100:00100000:3.0:1789762739.310753:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014904448:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762739.310789:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e37 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.310803:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762739.310807:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762739.310811:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e37 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.310835:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e37 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.313687:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e37 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.313703:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e37 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.313711:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e37 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.313716:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfb7b800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762739.313720:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e37 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.313727:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e37 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.313741:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e37 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.313746:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02e37 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762739.313790:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014904448:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3045us (97835us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762739.313797:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 172 00000100:00100000:3.0:1789762739.313804:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262614514166 time 262614625214 state 0x2 ref 0 00000100:00100000:0.0:1789762739.315723:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014904576 00000100:00100000:0.0:1789762739.315731:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014904576 00000100:00100000:0.0:1789762739.315767:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262614514166 time 262614625214 state 0x0 ref 1 00000100:00100000:0.0:1789762739.315779:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262714625214 time 262614625214 state 0x1 ref 1 00000001:00080000:2.0:1789762739.325871:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967439 is committed 00000100:00100000:0.0:1789762739.410801:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262714625214 time 262714695179 state 0x0 ref 1 00000100:00100000:0.0:1789762739.410845:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014904576:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762739.410905:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e3e lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.410924:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762739.410928:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762739.410934:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e3e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.410953:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e3e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.414252:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e3e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.414263:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e3e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.414271:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e3e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.414277:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9954fe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762739.414280:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e3e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.414299:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e3e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.414315:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e3e lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.414321:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e3e lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762739.414368:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014904576:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3535us (98704us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762739.414375:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 173 00000100:00100000:0.0:1789762739.414383:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262714625214 time 262714695179 state 0x2 ref 0 00000100:00100000:2.0:1789762739.416172:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762739.416207:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014904704 00000100:00100000:0.0:1789762739.416214:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014904704 00000100:00100000:0.0:1789762739.416246:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262714625214 time 262714695179 state 0x0 ref 1 00000100:00100000:0.0:1789762739.416256:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262814695179 time 262714695179 state 0x1 ref 1 00000100:00100000:3.0:1789762739.510895:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262814695179 time 262814791181 state 0x0 ref 1 00000100:00100000:3.0:1789762739.510931:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf513480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014904704:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762739.510967:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e45 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.510981:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762739.510985:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762739.510989:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e45 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.511000:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e45 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.514011:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e45 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.514021:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e45 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.514029:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e45 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.514035:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762739.514038:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e45 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.514045:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e45 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.514066:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e45 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.514072:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e45 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762739.514146:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf513480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014904704:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3220us (97973us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762739.514153:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 174 00000100:00100000:3.0:1789762739.514161:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262814695179 time 262814791181 state 0x2 ref 0 00000100:00100000:3.0:1789762739.516566:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762739.516614:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014904832 00000100:00100000:0.0:1789762739.516623:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014904832 00000100:00100000:0.0:1789762739.516665:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262814695179 time 262814791181 state 0x0 ref 1 00000100:00100000:0.0:1789762739.516680:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262914791181 time 262814791181 state 0x1 ref 1 00000100:00100000:0.0:1789762739.610951:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262914791181 time 262914848609 state 0x0 ref 1 00000100:00100000:0.0:1789762739.610971:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf566680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014904832:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762739.611005:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e4c lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.611018:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762739.611021:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762739.611026:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e4c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.611037:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e4c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.614021:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e4c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.614032:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e4c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.614039:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e4c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.614045:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9954fe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762739.614049:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e4c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.614062:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e4c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.614075:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e4c lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.614080:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ca9ed129/0x416193e012b02e4c lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762739.614143:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf566680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014904832:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3178us (97577us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762739.614150:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 175 00000100:00100000:0.0:1789762739.614158:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262914791181 time 262914848609 state 0x2 ref 0 00000100:00100000:0.0:1789762739.616260:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014904960 00000100:00100000:0.0:1789762739.616268:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014904960 00000100:00100000:0.0:1789762739.616300:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 262914791181 time 262914848609 state 0x0 ref 1 00000100:00100000:0.0:1789762739.616310:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263014848609 time 262914848609 state 0x1 ref 1 00000100:00100000:3.0:1789762739.711133:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263014848609 time 263015028024 state 0x0 ref 1 00000100:00100000:3.0:1789762739.711158:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014904960:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762739.711195:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e53 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.711210:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762739.711214:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762739.711219:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e53 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.711232:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e53 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.714428:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e53 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.714439:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e53 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.714446:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e53 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.714452:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762739.714455:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e53 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.714462:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e53 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.714476:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e53 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.714481:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e53 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762739.714528:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014904960:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3380us (98307us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762739.714536:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 176 00000100:00100000:3.0:1789762739.714549:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263014848609 time 263015028024 state 0x2 ref 0 00000100:00100000:0.0:1789762739.716556:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014905088 00000100:00100000:0.0:1789762739.716564:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014905088 00000100:00100000:0.0:1789762739.716600:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263014848609 time 263015028024 state 0x0 ref 1 00000100:00100000:0.0:1789762739.716610:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263115028024 time 263015028024 state 0x1 ref 1 00000100:00100000:0.0:1789762739.811287:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263115028024 time 263115161637 state 0x0 ref 1 00000100:00100000:0.0:1789762739.811307:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014905088:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762739.811335:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002da51bb6/0x416193e012b02e5a lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.811371:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762739.811377:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762739.811384:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002da51bb6/0x416193e012b02e5a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762739.811421:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002da51bb6/0x416193e012b02e5a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.814654:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002da51bb6/0x416193e012b02e5a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.814676:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002da51bb6/0x416193e012b02e5a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.814684:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002da51bb6/0x416193e012b02e5a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.814689:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf586400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762739.814715:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002da51bb6/0x416193e012b02e5a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.814727:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002da51bb6/0x416193e012b02e5a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.814747:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002da51bb6/0x416193e012b02e5a lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762739.814758:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002da51bb6/0x416193e012b02e5a lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762739.814879:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014905088:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3578us (98403us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762739.814886:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 177 00000100:00100000:2.0:1789762739.814893:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263115028024 time 263115161637 state 0x2 ref 0 00000100:00100000:0.0:1789762739.816933:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014905216 00000100:00100000:0.0:1789762739.816941:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014905216 00000100:00100000:0.0:1789762739.816999:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263115028024 time 263115161637 state 0x0 ref 1 00000100:00100000:0.0:1789762739.817018:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263215161637 time 263115161637 state 0x1 ref 1 00000100:00100000:3.0:1789762739.911445:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263215161637 time 263215329856 state 0x0 ref 1 00000100:00100000:3.0:1789762739.911501:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014905216:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762739.911553:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e61 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.911576:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762739.911582:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762739.911589:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e61 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.911619:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e61 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.914709:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e61 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.914719:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e61 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.914727:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e61 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.914733:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762739.914745:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e61 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.914752:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e61 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.914767:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e61 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762739.914772:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e61 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762739.914834:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014905216:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3367us (97948us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762739.914859:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 178 00000100:00100000:3.0:1789762739.914873:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263215161637 time 263215329856 state 0x2 ref 0 00000100:00100000:0.0:1789762739.916919:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762740.011665:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263315329856 time 263315554454 state 0x0 ref 1 00000100:00100000:2.0:1789762740.011711:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014905344:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762740.011757:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ccb06bd9/0x416193e012b02e68 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762740.011777:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762740.011783:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762740.011791:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ccb06bd9/0x416193e012b02e68 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762740.011836:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ccb06bd9/0x416193e012b02e68 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.015400:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ccb06bd9/0x416193e012b02e68 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.015411:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ccb06bd9/0x416193e012b02e68 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.015424:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ccb06bd9/0x416193e012b02e68 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.015429:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfb7b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762740.015432:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ccb06bd9/0x416193e012b02e68 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.015438:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ccb06bd9/0x416193e012b02e68 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.015451:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ccb06bd9/0x416193e012b02e68 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.015457:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ccb06bd9/0x416193e012b02e68 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762740.015504:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014905344:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3803us (98591us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762740.015512:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 179 00000100:00100000:0.0:1789762740.015520:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263315329856 time 263315554454 state 0x2 ref 0 00000100:00100000:0.0:1789762740.017498:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762740.017544:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014905472 00000100:00100000:2.0:1789762740.017552:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014905472 00000100:00100000:2.0:1789762740.017583:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263315329856 time 263315554454 state 0x0 ref 1 00000100:00100000:2.0:1789762740.017593:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263415554454 time 263315554454 state 0x1 ref 1 00000100:00100000:3.0:1789762740.111765:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263415554454 time 263415659188 state 0x0 ref 1 00000100:00100000:3.0:1789762740.111805:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014905472:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762740.111869:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e6f lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.111890:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762740.111896:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762740.111904:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e6f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.111938:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e6f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.115411:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e6f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.115422:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e6f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.115429:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e6f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.115434:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762740.115439:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e6f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.115445:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e6f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.115457:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e6f lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.115463:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02e6f lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762740.115518:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014905472:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3736us (98019us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762740.115525:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 180 00000100:00100000:3.0:1789762740.115531:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263415554454 time 263415659188 state 0x2 ref 0 00000100:00100000:0.0:1789762740.117524:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014905600 00000100:00100000:0.0:1789762740.117532:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014905600 00000100:00100000:0.0:1789762740.117565:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263415554454 time 263415659188 state 0x0 ref 1 00000100:00100000:0.0:1789762740.117582:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263515659188 time 263415659188 state 0x1 ref 1 00010000:00010000:0.0:1789762740.151206:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762740.211865:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263515659188 time 263515760729 state 0x0 ref 1 00000100:00100000:0.0:1789762740.211906:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014905600:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762740.211952:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e76 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.211967:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762740.211972:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762740.211977:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e76 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.211990:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e76 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.215603:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e76 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.215614:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e76 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.215622:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e76 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.215646:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe58800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762740.215651:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e76 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.215659:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e76 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.215674:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e76 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.215679:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e76 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762740.215759:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014905600:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3858us (98281us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762740.215772:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 181 00000100:00100000:0.0:1789762740.215786:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263515659188 time 263515760729 state 0x2 ref 0 00000100:00100000:2.0:1789762740.217759:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762740.217823:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014905728 00000100:00100000:0.0:1789762740.217832:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014905728 00000100:00100000:0.0:1789762740.217882:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263515659188 time 263515760729 state 0x0 ref 1 00000100:00100000:0.0:1789762740.217892:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263615760729 time 263515760729 state 0x1 ref 1 00000100:00100000:3.0:1789762740.311975:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263615760729 time 263615871209 state 0x0 ref 1 00000100:00100000:3.0:1789762740.312000:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf510700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014905728:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762740.312037:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e7d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.312050:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762740.312054:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762740.312059:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e7d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.312069:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e7d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.316223:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e7d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.316237:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e7d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.316262:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e7d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.316271:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfb7a000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762740.316275:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e7d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.316282:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e7d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.316297:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e7d lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.316302:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e7d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762740.316365:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf510700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014905728:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4368us (98600us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762740.316373:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 182 00000100:00100000:3.0:1789762740.316383:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263615760729 time 263615871209 state 0x2 ref 0 00000100:00100000:3.0:1789762740.318441:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762740.318499:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014905856 00000100:00100000:0.0:1789762740.318517:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014905856 00000100:00100000:0.0:1789762740.318571:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263615760729 time 263615871209 state 0x0 ref 1 00000100:00100000:0.0:1789762740.318588:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263715871209 time 263615871209 state 0x1 ref 1 00000100:00100000:0.0:1789762740.412036:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263715871209 time 263715931376 state 0x0 ref 1 00000100:00100000:0.0:1789762740.412059:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf567100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014905856:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762740.412156:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e84 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.412170:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762740.412174:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762740.412179:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e84 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.412198:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e84 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.415341:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e84 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.415356:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e84 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.415372:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e84 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.415390:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe58800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762740.415395:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e84 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.415401:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e84 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.415415:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e84 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.415420:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e84 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762740.415473:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf567100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014905856:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3417us (97028us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762740.415480:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 183 00000100:00100000:0.0:1789762740.415488:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263715871209 time 263715931376 state 0x2 ref 0 00000100:00100000:3.0:1789762740.417335:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762740.417368:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014905984 00000100:00100000:0.0:1789762740.417375:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014905984 00000100:00100000:0.0:1789762740.417407:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263715871209 time 263715931376 state 0x0 ref 1 00000100:00100000:0.0:1789762740.417417:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263815931376 time 263715931376 state 0x1 ref 1 00000100:00100000:3.0:1789762740.512140:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263815931376 time 263816036606 state 0x0 ref 1 00000100:00100000:3.0:1789762740.512179:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014905984:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762740.512224:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e8b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.512238:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762740.512242:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762740.512247:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e8b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.512258:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e8b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.515407:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e8b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.515416:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e8b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.515422:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e8b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.515427:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfb7a000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762740.515431:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e8b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.515437:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e8b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.515457:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e8b lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.515463:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000223fb4f7/0x416193e012b02e8b lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762740.515509:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf565180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014905984:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3337us (98175us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762740.515515:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 184 00000100:00100000:3.0:1789762740.515522:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263815931376 time 263816036606 state 0x2 ref 0 00000100:00100000:3.0:1789762740.517250:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762740.517298:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014906112 00000100:00100000:0.0:1789762740.517312:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014906112 00000100:00100000:0.0:1789762740.517346:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263815931376 time 263816036606 state 0x0 ref 1 00000100:00100000:0.0:1789762740.517364:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263916036606 time 263816036606 state 0x1 ref 1 00000100:00100000:0.0:1789762740.612204:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263916036606 time 263916096156 state 0x0 ref 1 00000100:00100000:0.0:1789762740.612233:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014906112:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762740.612304:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e92 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.612325:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762740.612330:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762740.612338:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e92 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.612367:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e92 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.616538:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e92 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.616554:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e92 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.616563:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e92 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.616575:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe58800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762740.616584:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e92 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.616592:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e92 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.616609:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e92 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.616632:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042fb4ef0/0x416193e012b02e92 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762740.616695:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014906112:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4469us (99443us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762740.616706:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 185 00000100:00100000:0.0:1789762740.616716:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263916036606 time 263916096156 state 0x2 ref 0 00000100:00100000:0.0:1789762740.619035:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014906240 00000100:00100000:0.0:1789762740.619044:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014906240 00000100:00100000:0.0:1789762740.619083:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 263916036606 time 263916096156 state 0x0 ref 1 00000100:00100000:0.0:1789762740.619130:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264016096156 time 263916096156 state 0x1 ref 1 00000100:00100000:3.0:1789762740.712299:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264016096156 time 264016198338 state 0x0 ref 1 00000100:00100000:3.0:1789762740.712321:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014906240:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762740.712356:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02e99 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.712370:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762740.712374:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762740.712379:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02e99 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.712392:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02e99 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.715893:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02e99 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.715903:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02e99 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.715910:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02e99 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.715914:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fde600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762740.715918:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02e99 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.715924:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02e99 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.715936:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02e99 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.715941:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02e99 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762740.715985:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014906240:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3670us (96995us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762740.715992:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 186 00000100:00100000:3.0:1789762740.716005:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264016096156 time 264016198338 state 0x2 ref 0 00000100:00100000:3.0:1789762740.717694:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762740.717735:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014906368 00000100:00100000:0.0:1789762740.717767:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014906368 00000100:00100000:0.0:1789762740.717862:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264016096156 time 264016198338 state 0x0 ref 1 00000100:00100000:0.0:1789762740.717874:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264116198338 time 264016198338 state 0x1 ref 1 00000100:00100000:0.0:1789762740.812393:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264116198338 time 264116285568 state 0x0 ref 1 00000100:00100000:0.0:1789762740.812416:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014906368:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762740.812448:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b02ea0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.812462:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762740.812466:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762740.812470:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b02ea0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762740.812482:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b02ea0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762740.818592:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762740.912485:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264216285568 time 264216380489 state 0x0 ref 1 00000100:00100000:3.0:1789762740.912508:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014906496:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762740.912544:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02ea7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.912558:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762740.912562:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762740.912566:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02ea7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.912577:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02ea7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.916454:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02ea7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.916464:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02ea7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.916471:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02ea7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.916490:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87fde600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762740.916494:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02ea7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.916500:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02ea7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.916513:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02ea7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762740.916518:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02ea7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762740.916562:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014906496:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4063us (97973us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762740.916569:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 188 00000100:00100000:3.0:1789762740.916577:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264216285568 time 264216380489 state 0x2 ref 0 00000100:00100000:3.0:1789762740.918672:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762740.918722:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014906624 00000100:00100000:0.0:1789762740.918730:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014906624 00000100:00100000:0.0:1789762740.918766:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264216285568 time 264216380489 state 0x0 ref 1 00000100:00100000:0.0:1789762740.918776:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264316380489 time 264216380489 state 0x1 ref 1 00000100:00080000:0.0:1789762741.012625:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762741.016245:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cd0488a8/0x416193e012b02eae lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.016256:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd0488a8/0x416193e012b02eae lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.016263:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd0488a8/0x416193e012b02eae lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.016269:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf4f3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762741.016272:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd0488a8/0x416193e012b02eae lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.016279:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd0488a8/0x416193e012b02eae lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.016293:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd0488a8/0x416193e012b02eae lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.016298:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd0488a8/0x416193e012b02eae lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762741.016346:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014906624:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3761us (97675us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762741.016354:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 189 00000100:00100000:0.0:1789762741.016368:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264316380489 time 264316451539 state 0x2 ref 0 00000100:00100000:2.0:1789762741.018556:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762741.112665:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264416451539 time 264416561314 state 0x0 ref 1 00000100:00100000:3.0:1789762741.112691:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf511c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014906752:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762741.112728:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02eb5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.112756:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762741.112760:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762741.112765:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02eb5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.112776:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009414822d/0x416193e012b02eb5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762741.117604:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762741.117658:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014906880 00000100:00100000:0.0:1789762741.117666:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014906880 00000100:00100000:0.0:1789762741.117704:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264416451539 time 264416561314 state 0x0 ref 1 00000100:00100000:0.0:1789762741.117715:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264516561314 time 264416561314 state 0x1 ref 1 00010000:00010000:2.0:1789762741.175238:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762741.212711:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264516561314 time 264516608606 state 0x0 ref 1 00000100:00100000:0.0:1789762741.212733:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf511f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014906880:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762741.212770:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b02ebc lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.212784:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762741.212787:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762741.212792:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b02ebc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.212815:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b02ebc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.215905:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b02ebc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.215916:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b02ebc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.215933:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b02ebc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.215939:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762741.215943:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b02ebc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.215949:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b02ebc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.215964:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b02ebc lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.215970:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b02ebc lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762741.216016:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf511f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014906880:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3291us (98413us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762741.216023:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 191 00000100:00100000:0.0:1789762741.216031:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264516561314 time 264516608606 state 0x2 ref 0 00000100:00100000:3.0:1789762741.217695:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:3.0:1789762741.316071:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02ec3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.316082:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02ec3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.316107:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02ec3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.316114:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfec9e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762741.316118:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02ec3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.316124:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02ec3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.316138:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02ec3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.316151:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009057b1b5/0x416193e012b02ec3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762741.316207:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014907008:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3381us (98511us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762741.316219:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 192 00000100:00100000:3.0:1789762741.316226:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264616608606 time 264616678186 state 0x2 ref 0 00000100:00100000:3.0:1789762741.318013:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762741.318060:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014907136 00000100:00100000:0.0:1789762741.318069:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014907136 00000100:00100000:0.0:1789762741.318130:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264616608606 time 264616678186 state 0x0 ref 1 00000100:00100000:0.0:1789762741.318150:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264716678186 time 264616678186 state 0x1 ref 1 00000100:00100000:0.0:1789762741.346383:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014907392 00000100:00100000:0.0:1789762741.346396:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014907392 00000100:00100000:2.0:1789762741.346416:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014907648 00000100:00100000:3.0:1789762741.346417:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014907264 00000100:00100000:2.0:1789762741.346424:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014907648 00000100:00100000:3.0:1789762741.346425:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014907264 00000100:00100000:0.0:1789762741.346469:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 178 00000100:00100000:2.0:1789762741.346478:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 179 00000100:00100000:0.0:1789762741.346483:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702014907392:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762741.346486:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 276 00000100:00100000:2.0:1789762741.346494:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702014907648:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762741.346502:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014907264:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762741.346563:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702014907392:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 78us (257us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762741.346563:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014907264:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 58us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762741.346564:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702014907648:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 73us (222us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762741.346585:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 276 00000100:00100000:0.0:1789762741.346588:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 178 00000100:00100000:2.0:1789762741.346599:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 179 00000100:00100000:2.0:1789762741.367314:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022149504 00000100:00100000:2.0:1789762741.367321:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022149504 00000100:00080000:0.0:1789762741.367348:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762741.367371:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 77 00000100:00100000:2.0:1789762741.367383:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022149504:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762741.367409:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762741.367450:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022149504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 71us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762741.367455:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 77 00000100:00100000:0.0:1789762741.412851:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264716678186 time 264716746658 state 0x0 ref 1 00000100:00100000:0.0:1789762741.412874:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014907136:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762741.412911:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02eca lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.412928:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762741.412933:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762741.412940:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02eca lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.412956:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02eca lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.416432:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02eca lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.416455:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02eca lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.416462:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02eca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.416468:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762741.416473:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02eca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.416479:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02eca lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.416492:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02eca lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.416497:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02eca lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762741.416543:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014907136:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3678us (98530us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762741.416551:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 193 00000100:00100000:0.0:1789762741.416558:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264716678186 time 264716746658 state 0x2 ref 0 00000100:00100000:3.0:1789762741.418440:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014907776 00000100:00100000:3.0:1789762741.418447:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014907776 00000100:00100000:3.0:1789762741.418482:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264716678186 time 264716746658 state 0x0 ref 1 00000100:00100000:3.0:1789762741.418493:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264816746658 time 264716746658 state 0x1 ref 1 00000100:00100000:3.0:1789762741.512923:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264816746658 time 264816815581 state 0x0 ref 1 00000100:00100000:3.0:1789762741.512969:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014907776:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762741.513009:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02ed1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.513025:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762741.513030:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762741.513036:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02ed1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.513050:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02ed1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.516164:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02ed1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.516180:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02ed1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.516191:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02ed1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.516200:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8812c400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762741.516205:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02ed1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.516216:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02ed1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.516245:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02ed1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.516254:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02ed1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762741.516310:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014907776:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3348us (97903us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762741.516321:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 194 00000100:00100000:3.0:1789762741.516332:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264816746658 time 264816815581 state 0x2 ref 0 00000100:00100000:3.0:1789762741.518068:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762741.518132:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014907904 00000100:00100000:0.0:1789762741.518150:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014907904 00000100:00100000:0.0:1789762741.518194:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264816746658 time 264816815581 state 0x0 ref 1 00000100:00100000:0.0:1789762741.518206:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264916815581 time 264816815581 state 0x1 ref 1 00000100:00100000:0.0:1789762741.612975:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264916815581 time 264916871466 state 0x0 ref 1 00000100:00100000:0.0:1789762741.613002:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014907904:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762741.613039:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ed8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.613054:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762741.613057:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762741.613062:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ed8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.613073:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ed8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.616132:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ed8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.616141:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ed8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.616148:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ed8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.616170:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762741.616176:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ed8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.616187:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ed8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.616205:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ed8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.616214:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ed8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762741.616261:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014907904:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3270us (98195us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762741.616268:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 195 00000100:00100000:0.0:1789762741.616275:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 264916815581 time 264916871466 state 0x2 ref 0 00000100:00100000:3.0:1789762741.618125:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762741.713055:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265016871466 time 265016945408 state 0x0 ref 1 00000100:00100000:3.0:1789762741.713083:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014908032:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762741.713143:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02edf lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.713157:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762741.713161:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762741.713166:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02edf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.713191:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000029e24720/0x416193e012b02edf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762741.718457:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014908160 00000100:00100000:0.0:1789762741.718467:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014908160 00000100:00100000:0.0:1789762741.718512:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265016871466 time 265016945408 state 0x0 ref 1 00000100:00100000:0.0:1789762741.718529:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265116945408 time 265016945408 state 0x1 ref 1 00000100:00100000:0.0:1789762741.813152:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265116945408 time 265117046874 state 0x0 ref 1 00000100:00100000:0.0:1789762741.813186:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014908160:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762741.813219:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ee6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.813234:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762741.813238:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762741.813242:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ee6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.813254:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ee6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.816731:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ee6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.816743:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ee6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.816750:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ee6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.816756:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762741.816760:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ee6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.816767:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ee6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.816791:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ee6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762741.816811:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ee6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762741.816882:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014908160:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3701us (98512us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762741.816890:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 197 00000100:00100000:0.0:1789762741.816899:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265116945408 time 265117046874 state 0x2 ref 0 00000100:00100000:3.0:1789762741.818989:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014908288 00000100:00100000:3.0:1789762741.818998:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014908288 00000100:00100000:3.0:1789762741.819035:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265116945408 time 265117046874 state 0x0 ref 1 00000100:00100000:3.0:1789762741.819046:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265217046874 time 265117046874 state 0x1 ref 1 00000100:00100000:3.0:1789762741.879263:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022150144:0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.879277:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022149760:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.879303:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022149888:0@lo:400:kworker.0 00000100:00100000:3.0:1789762741.879313:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762741.879331:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762741.879369:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762741.879393:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022150272:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.879402:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022150400:0@lo:400:kworker.0 00000100:00100000:3.0:1789762741.879411:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022149760 00000100:00100000:3.0:1789762741.879419:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022149760 00000100:00100000:0.0:1789762741.879442:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762741.879449:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022149888 00000100:00100000:2.0:1789762741.879455:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022149888 00000100:00100000:3.0:1789762741.879472:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 277 00000100:00100000:3.0:1789762741.879486:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dbc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022149760:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.879500:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022150016 00000100:00100000:2.0:1789762741.879505:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:0.0:1789762741.879506:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022150016 00000100:00100000:2.0:1789762741.879521:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022149888:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762741.879539:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dbc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022149760:12345-0@lo:400:kworker.0 Request processed in 59us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762741.879547:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 277 00000100:00100000:0.0:1789762741.879559:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:0.0:1789762741.879579:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022150016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.879579:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022149888:12345-0@lo:400:kworker.0 Request processed in 60us (205us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762741.879586:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022150144 00000100:00100000:3.0:1789762741.879592:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022150144 00000100:00100000:2.0:1789762741.879597:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:3.0:1789762741.879636:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022150400 00000100:00100000:0.0:1789762741.879637:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022150016:12345-0@lo:400:kworker.0 Request processed in 65us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762741.879640:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022150400 00000100:00100000:0.0:1789762741.879650:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:2.0:1789762741.879650:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 88 00000100:00100000:2.0:1789762741.879667:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022150144:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762741.879677:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 89 00000100:00100000:3.0:1789762741.879690:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022150400:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762741.879694:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762741.879700:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762741.879714:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022150144:12345-0@lo:400:kworker.0 Request processed in 53us (402us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762741.879734:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 88 00000100:00100000:3.0:1789762741.879739:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022150400:12345-0@lo:400:kworker.0 Request processed in 51us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762741.879752:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022150144:0@lo:400:kworker.0 00000100:00100000:3.0:1789762741.879759:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 89 00000100:00100000:2.0:1789762741.879771:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762741.879815:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022149888:0@lo:400:kworker.0 00000100:00100000:2.0:1789762741.879835:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022149760:0@lo:400:kworker.0 00000100:00100000:0.0:1789762741.879837:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecdf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022150400:0@lo:400:kworker.0 00000100:00080000:0.0:1789762741.879875:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762741.879946:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022150272:0@lo:400:kworker.0 00010000:00010000:3.0:1789762741.916451:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b02eed lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.916462:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b02eed lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.916469:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b02eed lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.916475:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762741.916479:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b02eed lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.916485:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b02eed lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.916500:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b02eed lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762741.916505:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b02eed lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762741.916549:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014908288:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3298us (97607us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762741.916556:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 198 00000100:00100000:3.0:1789762741.916571:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265217046874 time 265217136893 state 0x2 ref 0 00000100:00100000:3.0:1789762741.918265:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762741.918316:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014908416 00000100:00100000:0.0:1789762741.918324:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014908416 00000100:00100000:0.0:1789762741.918355:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265217046874 time 265217136893 state 0x0 ref 1 00000100:00100000:0.0:1789762741.918367:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265317136893 time 265217136893 state 0x1 ref 1 00000100:00100000:0.0:1789762742.013345:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265317136893 time 265317222140 state 0x0 ref 1 00000100:00100000:0.0:1789762742.013370:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014908416:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762742.013409:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ef4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.013425:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.013429:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.013434:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ef4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.013450:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ef4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.016990:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ef4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.017014:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ef4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.017024:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ef4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.017032:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762742.017037:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ef4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.017046:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ef4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.017065:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ef4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.017073:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b02ef4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762742.017172:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014908416:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3809us (98906us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762742.017181:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 199 00000100:00100000:0.0:1789762742.017192:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265317136893 time 265317222140 state 0x2 ref 0 00000100:00100000:2.0:1789762742.019435:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762742.019503:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014908544 00000100:00100000:3.0:1789762742.019512:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014908544 00000100:00100000:3.0:1789762742.019550:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265317136893 time 265317222140 state 0x0 ref 1 00000100:00100000:3.0:1789762742.019562:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265417222140 time 265317222140 state 0x1 ref 1 00000100:00100000:3.0:1789762742.113392:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265417222140 time 265417283241 state 0x0 ref 1 00000100:00100000:3.0:1789762742.113418:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf513b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014908544:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762742.113454:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02efb lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.113468:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762742.113480:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762742.113485:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02efb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.113497:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02efb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.116595:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02efb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.116614:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02efb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.116621:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02efb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.116627:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe59600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762742.116631:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02efb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.116638:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02efb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.116651:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02efb lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.116657:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02efb lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762742.116702:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf513b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014908544:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3295us (97270us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762742.116709:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 200 00000100:00100000:3.0:1789762742.116715:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265417222140 time 265417283241 state 0x2 ref 0 00000100:00100000:0.0:1789762742.118640:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014908672 00000100:00100000:0.0:1789762742.118654:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014908672 00000100:00100000:0.0:1789762742.118690:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265417222140 time 265417283241 state 0x0 ref 1 00000100:00100000:0.0:1789762742.118701:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265517283241 time 265417283241 state 0x1 ref 1 00010000:00010000:0.0:1789762742.199176:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762742.213437:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265517283241 time 265517336166 state 0x0 ref 1 00000100:00100000:0.0:1789762742.213458:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014908672:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762742.213493:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f02 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.213507:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.213511:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.213516:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f02 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.213526:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f02 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.216428:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f02 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.216439:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f02 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.216447:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f02 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.216460:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf587a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762742.216464:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f02 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.216477:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f02 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.216492:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f02 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.216498:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f02 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762742.216543:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014908672:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3092us (97954us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762742.216550:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 201 00000100:00100000:0.0:1789762742.216559:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265517283241 time 265517336166 state 0x2 ref 0 00000100:00100000:0.0:1789762742.218365:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762742.218415:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014908800 00000100:00100000:3.0:1789762742.218431:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014908800 00000100:00100000:3.0:1789762742.218474:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265517283241 time 265517336166 state 0x0 ref 1 00000100:00100000:3.0:1789762742.218488:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265617336166 time 265517336166 state 0x1 ref 1 00000100:00100000:3.0:1789762742.313571:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265617336166 time 265617464268 state 0x0 ref 1 00000100:00100000:3.0:1789762742.313604:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014908800:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762742.313651:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02f09 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.313668:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762742.313690:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762742.313697:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02f09 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.313714:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02f09 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.317361:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02f09 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.317373:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02f09 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.317381:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02f09 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.317395:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe59600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762742.317399:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02f09 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.317406:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02f09 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.317421:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02f09 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.317428:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c91f6f15/0x416193e012b02f09 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762742.317493:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014908800:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3903us (99126us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762742.317504:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 202 00000100:00100000:3.0:1789762742.317516:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265617336166 time 265617464268 state 0x2 ref 0 00000100:00100000:2.0:1789762742.319424:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762742.319481:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014908928 00000100:00100000:0.0:1789762742.319490:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014908928 00000100:00100000:0.0:1789762742.319526:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265617336166 time 265617464268 state 0x0 ref 1 00000100:00100000:0.0:1789762742.319538:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265717464268 time 265617464268 state 0x1 ref 1 00000100:00100000:0.0:1789762742.413698:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265717464268 time 265717586437 state 0x0 ref 1 00000100:00100000:0.0:1789762742.413732:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf511500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014908928:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762742.413780:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f10 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.413823:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.413829:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.413840:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f10 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.413863:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f10 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.418325:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f10 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.418336:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f10 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.418346:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f10 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.418357:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf587a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762742.418364:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f10 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.418392:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f10 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.418415:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f10 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.418423:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f10 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762742.418503:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf511500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014908928:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4784us (99078us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762742.418516:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 203 00000100:00100000:0.0:1789762742.418544:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265717464268 time 265717586437 state 0x2 ref 0 00000100:00100000:3.0:1789762742.420836:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014909056 00000100:00100000:3.0:1789762742.420847:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014909056 00000100:00100000:3.0:1789762742.420892:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265717464268 time 265717586437 state 0x0 ref 1 00000100:00100000:3.0:1789762742.420925:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265817586437 time 265717586437 state 0x1 ref 1 00000100:00100000:3.0:1789762742.513764:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265817586437 time 265817660546 state 0x0 ref 1 00000100:00100000:3.0:1789762742.513816:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014909056:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762742.513853:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f17 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.513869:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762742.513872:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762742.513877:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f17 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.513889:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f17 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.516960:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f17 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.516970:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f17 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.516976:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f17 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.516982:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb8600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762742.516985:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f17 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.516992:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f17 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.517013:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f17 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.517019:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f17 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762742.517064:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014909056:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3257us (96300us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762742.517072:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 204 00000100:00100000:3.0:1789762742.517079:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265817586437 time 265817660546 state 0x2 ref 0 00000100:00100000:3.0:1789762742.519191:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762742.519266:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014909184 00000100:00100000:0.0:1789762742.519274:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014909184 00000100:00100000:0.0:1789762742.519311:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265817586437 time 265817660546 state 0x0 ref 1 00000100:00100000:0.0:1789762742.519322:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265917660546 time 265817660546 state 0x1 ref 1 00000100:00100000:0.0:1789762742.613877:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265917660546 time 265917772766 state 0x0 ref 1 00000100:00100000:0.0:1789762742.613900:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014909184:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762742.613935:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f1e lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.613960:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.613963:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.613968:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f1e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.613979:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f1e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.617049:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f1e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.617062:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f1e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.617071:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f1e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.617080:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf587a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762742.617103:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f1e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.617110:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f1e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.617124:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f1e lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.617140:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f1e lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762742.617182:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014909184:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3292us (97992us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762742.617189:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 205 00000100:00100000:0.0:1789762742.617196:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 265917660546 time 265917772766 state 0x2 ref 0 00000100:00100000:3.0:1789762742.619232:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762742.713994:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266017772766 time 266017885642 state 0x0 ref 1 00000100:00100000:3.0:1789762742.714022:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014909312:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762742.714066:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f25 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.714111:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762742.714117:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762742.714123:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f25 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.714162:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006907e820/0x416193e012b02f25 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762742.720900:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014909440 00000100:00100000:0.0:1789762742.720915:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014909440 00000100:00100000:0.0:1789762742.720975:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266017772766 time 266017885642 state 0x0 ref 1 00000100:00100000:0.0:1789762742.720995:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266117885642 time 266017885642 state 0x1 ref 1 00000100:00100000:0.0:1789762742.814187:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266117885642 time 266118073108 state 0x0 ref 1 00000100:00100000:0.0:1789762742.814225:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014909440:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762742.814279:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f2c lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.814312:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.814316:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762742.814320:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f2c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.814348:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f2c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.818561:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f2c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.818571:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f2c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.818579:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f2c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.818584:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf587a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762742.818588:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f2c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.818594:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f2c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.818607:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f2c lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762742.818613:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f2c lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762742.818669:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014909440:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4462us (97900us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762742.818677:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 207 00000100:00100000:0.0:1789762742.818685:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266117885642 time 266118073108 state 0x2 ref 0 00000100:00100000:3.0:1789762742.820689:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014909568 00000100:00100000:3.0:1789762742.820697:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014909568 00000100:00100000:3.0:1789762742.820731:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266117885642 time 266118073108 state 0x0 ref 1 00000100:00100000:3.0:1789762742.820742:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266218073108 time 266118073108 state 0x1 ref 1 00010000:00010000:3.0:1789762742.918529:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02f33 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.918556:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02f33 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.918591:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02f33 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.918598:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe59200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762742.918603:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02f33 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.918611:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02f33 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.918649:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02f33 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762742.918657:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000250936f6/0x416193e012b02f33 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762742.918710:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014909568:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4434us (98056us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762742.918719:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 208 00000100:00100000:3.0:1789762742.918728:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266218073108 time 266218156350 state 0x2 ref 0 00000100:00100000:3.0:1789762742.920877:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762742.920932:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014909696 00000100:00100000:0.0:1789762742.920942:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014909696 00000100:00100000:0.0:1789762742.920985:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266218073108 time 266218156350 state 0x0 ref 1 00000100:00100000:0.0:1789762742.920998:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266318156350 time 266218156350 state 0x1 ref 1 00000100:00100000:0.0:1789762743.014325:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266318156350 time 266318221676 state 0x0 ref 1 00000100:00100000:0.0:1789762743.014365:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014909696:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:2.0:1789762743.014391:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762743.014403:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f3a lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.014418:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.014422:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.014426:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f3a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.014438:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f3a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.018215:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f3a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.018226:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f3a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.018233:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f3a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.018238:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf587a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762743.018242:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f3a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.018249:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f3a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.018264:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f3a lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.018269:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f3a lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762743.018316:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014909696:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3976us (97443us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762743.018323:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 209 00000100:00100000:0.0:1789762743.018337:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266318156350 time 266318221676 state 0x2 ref 0 00000100:00100000:3.0:1789762743.020264:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762743.114427:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266418221676 time 266418316310 state 0x0 ref 1 00000100:00100000:3.0:1789762743.114449:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014909824:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762743.114487:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f41 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.114500:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762743.114504:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762743.114508:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f41 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.114533:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f41 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.117579:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f41 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.117589:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f41 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.117595:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f41 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.117600:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aac00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762743.117603:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f41 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.117609:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f41 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.117623:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f41 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.117628:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f41 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762743.117687:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014909824:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3245us (97423us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762743.117695:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 210 00000100:00100000:3.0:1789762743.117703:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266418221676 time 266418316310 state 0x2 ref 0 00000100:00100000:3.0:1789762743.119394:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762743.119471:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014909952 00000100:00100000:0.0:1789762743.119480:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014909952 00000100:00100000:0.0:1789762743.119516:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266418221676 time 266418316310 state 0x0 ref 1 00000100:00100000:0.0:1789762743.119527:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266518316310 time 266418316310 state 0x1 ref 1 00000100:00100000:0.0:1789762743.214494:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266518316310 time 266518391203 state 0x0 ref 1 00000100:00100000:0.0:1789762743.214520:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014909952:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762743.214556:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f48 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.214571:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.214574:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.214579:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f48 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.214590:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f48 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.217640:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f48 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.217655:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f48 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.217675:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f48 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.217683:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf587a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762743.217689:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f48 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.217699:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f48 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.217716:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f48 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.217724:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000fde4c55/0x416193e012b02f48 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762743.217796:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014909952:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3272us (98387us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762743.217806:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 211 00000100:00100000:0.0:1789762743.217817:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266518316310 time 266518391203 state 0x2 ref 0 00000100:00100000:2.0:1789762743.219595:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:3.0:1789762743.223179:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762743.314584:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266618391203 time 266618480646 state 0x0 ref 1 00000100:00100000:3.0:1789762743.314608:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014910080:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762743.314644:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f4f lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.314658:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762743.314661:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762743.314666:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f4f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.314677:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f4f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.317863:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f4f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.317892:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f4f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.317902:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f4f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.317910:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aac00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762743.317916:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f4f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.317927:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f4f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.317944:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f4f lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.317953:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f4f lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762743.318009:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf512300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014910080:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3408us (98413us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762743.318020:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 212 00000100:00100000:3.0:1789762743.318031:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266618391203 time 266618480646 state 0x2 ref 0 00000100:00100000:2.0:1789762743.319990:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762743.320056:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014910208 00000100:00100000:0.0:1789762743.320066:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014910208 00000100:00100000:0.0:1789762743.320135:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266618391203 time 266618480646 state 0x0 ref 1 00000100:00100000:0.0:1789762743.320163:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266718480646 time 266618480646 state 0x1 ref 1 00000100:00100000:0.0:1789762743.414688:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266718480646 time 266718580182 state 0x0 ref 1 00000100:00100000:0.0:1789762743.414717:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf511880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014910208:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762743.414760:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f56 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.414778:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.414795:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.414802:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f56 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.414817:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f56 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.417847:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f56 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.417861:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f56 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.417871:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f56 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.417901:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762743.417907:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f56 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.417918:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f56 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.417937:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f56 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.417946:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f56 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762743.418009:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf511880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014910208:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3303us (98019us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762743.418020:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 213 00000100:00100000:0.0:1789762743.418031:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266718480646 time 266718580182 state 0x2 ref 0 00000100:00100000:0.0:1789762743.419744:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762743.514880:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266818580182 time 266818775665 state 0x0 ref 1 00000100:00100000:3.0:1789762743.514918:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014910336:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762743.514955:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f5d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.514968:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762743.514972:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762743.514976:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f5d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.514987:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f5d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.518733:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f5d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.518745:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f5d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.518753:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f5d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.518760:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aac00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762743.518764:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f5d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.518772:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f5d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.518816:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f5d lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.518841:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b02f5d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762743.518897:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014910336:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3984us (99152us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762743.518907:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 214 00000100:00100000:3.0:1789762743.518918:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266818580182 time 266818775665 state 0x2 ref 0 00000100:00100000:0.0:1789762743.520844:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014910464 00000100:00100000:0.0:1789762743.520854:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014910464 00000100:00100000:0.0:1789762743.520898:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266818580182 time 266818775665 state 0x0 ref 1 00000100:00100000:0.0:1789762743.520914:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266918775665 time 266818775665 state 0x1 ref 1 00000100:00100000:0.0:1789762743.614932:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266918775665 time 266918827747 state 0x0 ref 1 00000100:00100000:0.0:1789762743.614957:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014910464:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762743.614993:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f64 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.615007:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.615010:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.615015:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f64 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.615025:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f64 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.617811:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f64 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.617821:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f64 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.617829:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f64 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.617835:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762743.617838:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f64 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.617853:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f64 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.617868:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f64 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.617873:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f64 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762743.617917:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014910464:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2969us (97184us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762743.617924:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 215 00000100:00100000:0.0:1789762743.617932:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266918775665 time 266918827747 state 0x2 ref 0 00000100:00100000:3.0:1789762743.619821:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014910592 00000100:00100000:3.0:1789762743.619830:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014910592 00000100:00100000:3.0:1789762743.619865:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 266918775665 time 266918827747 state 0x0 ref 1 00000100:00100000:3.0:1789762743.619879:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267018827747 time 266918827747 state 0x1 ref 1 00000100:00100000:3.0:1789762743.714997:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267018827747 time 267018894360 state 0x0 ref 1 00000100:00100000:3.0:1789762743.715021:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014910592:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762743.715056:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f6b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.715070:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762743.715112:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762743.715117:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f6b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.715131:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f6b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.717544:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f6b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.717553:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f6b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.717559:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f6b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.717565:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfec9400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762743.717568:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f6b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.717574:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f6b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.717588:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f6b lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.717593:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f6b lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762743.717649:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014910592:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2636us (97882us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762743.717656:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 216 00000100:00100000:3.0:1789762743.717664:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267018827747 time 267018894360 state 0x2 ref 0 00000100:00100000:3.0:1789762743.719523:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762743.719565:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014910720 00000100:00100000:0.0:1789762743.719574:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014910720 00000100:00100000:0.0:1789762743.719609:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267018827747 time 267018894360 state 0x0 ref 1 00000100:00100000:0.0:1789762743.719620:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267118894360 time 267018894360 state 0x1 ref 1 00000100:00100000:0.0:1789762743.815065:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267118894360 time 267118960678 state 0x0 ref 1 00000100:00100000:0.0:1789762743.815111:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014910720:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762743.815148:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f72 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.815175:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.815178:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762743.815183:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f72 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.815195:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f72 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.818222:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f72 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.818233:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f72 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.818240:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f72 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.818246:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762743.818250:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f72 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.818257:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f72 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.818271:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f72 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762743.818287:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f72 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762743.818335:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014910720:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3256us (98812us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762743.818342:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 217 00000100:00100000:0.0:1789762743.818349:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267118894360 time 267118960678 state 0x2 ref 0 00000100:00100000:3.0:1789762743.820165:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762743.915157:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267218960678 time 267219052757 state 0x0 ref 1 00000100:00100000:3.0:1789762743.915181:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014910848:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762743.915226:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f79 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.915240:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762743.915243:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762743.915248:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f79 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762743.915258:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b7e92a5/0x416193e012b02f79 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762743.919767:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014910976 00000100:00100000:0.0:1789762743.919775:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014910976 00000100:00100000:0.0:1789762743.919826:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267218960678 time 267219052757 state 0x0 ref 1 00000100:00100000:0.0:1789762743.919837:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267319052757 time 267219052757 state 0x1 ref 1 00000100:00100000:0.0:1789762744.015208:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267319052757 time 267319104463 state 0x0 ref 1 00000100:00100000:0.0:1789762744.015232:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014910976:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762744.015268:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f80 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.015282:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.015285:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.015290:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f80 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.015317:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f80 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.018543:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f80 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.018554:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f80 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.018562:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f80 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.018567:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762744.018571:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f80 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.018578:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f80 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.018596:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f80 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.018605:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f80 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762744.018671:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014910976:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3449us (98955us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762744.018679:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 219 00000100:00100000:0.0:1789762744.018687:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267319052757 time 267319104463 state 0x2 ref 0 00000100:00100000:3.0:1789762744.020833:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014911104 00000100:00100000:3.0:1789762744.020841:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014911104 00000100:00100000:3.0:1789762744.020877:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267319052757 time 267319104463 state 0x0 ref 1 00000100:00100000:3.0:1789762744.020894:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267419104463 time 267319104463 state 0x1 ref 1 00010000:00010000:3.0:1789762744.117988:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02f87 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.117999:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02f87 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.118006:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02f87 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.118012:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfb7b800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762744.118015:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02f87 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.118022:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02f87 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.118037:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02f87 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.118043:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ac213ca7/0x416193e012b02f87 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762744.118115:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014911104:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2739us (97314us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762744.118139:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 220 00000100:00100000:3.0:1789762744.118150:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267419104463 time 267419222250 state 0x2 ref 0 00000100:00100000:3.0:1789762744.120019:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762744.120072:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014911232 00000100:00100000:0.0:1789762744.120083:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014911232 00000100:00100000:0.0:1789762744.120157:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267419104463 time 267419222250 state 0x0 ref 1 00000100:00100000:0.0:1789762744.120168:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267519222250 time 267419222250 state 0x1 ref 1 00000100:00100000:0.0:1789762744.215417:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267519222250 time 267519303790 state 0x0 ref 1 00000100:00100000:0.0:1789762744.215475:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014911232:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762744.215543:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f8e lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.215563:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.215568:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.215575:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f8e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.215592:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f8e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.220338:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f8e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.220349:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f8e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.220357:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f8e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.220363:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762744.220384:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f8e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.220394:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f8e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.220424:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f8e lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.220433:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b02f8e lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762744.220519:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014911232:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5073us (100493us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762744.220528:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 221 00000100:00100000:0.0:1789762744.220564:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267519222250 time 267519303790 state 0x2 ref 0 00000100:00100000:2.0:1789762744.223341:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762744.223404:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014911360 00000100:00100000:3.0:1789762744.223416:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014911360 00000100:00100000:3.0:1789762744.223487:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267519222250 time 267519303790 state 0x0 ref 1 00000100:00100000:3.0:1789762744.223521:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267619303790 time 267519303790 state 0x1 ref 1 00010000:00010000:0.0:1789762744.247193:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762744.315463:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267619303790 time 267619358922 state 0x0 ref 1 00000100:00100000:3.0:1789762744.315489:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014911360:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762744.315525:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02f95 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.315541:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762744.315545:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762744.315550:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02f95 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.315561:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02f95 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.319635:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02f95 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.319663:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02f95 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.319672:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02f95 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.319697:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762744.319704:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02f95 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.319715:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02f95 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.319738:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02f95 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.319748:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02f95 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762744.319856:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014911360:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4376us (96514us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762744.319867:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 222 00000100:00100000:3.0:1789762744.319875:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267619303790 time 267619358922 state 0x2 ref 0 00000100:00100000:2.0:1789762744.322368:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762744.322421:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014911488 00000100:00100000:0.0:1789762744.322430:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014911488 00000100:00100000:0.0:1789762744.322469:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267619303790 time 267619358922 state 0x0 ref 1 00000100:00100000:0.0:1789762744.322481:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267719358922 time 267619358922 state 0x1 ref 1 00000100:00100000:0.0:1789762744.415575:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267719358922 time 267719462102 state 0x0 ref 1 00000100:00100000:0.0:1789762744.415613:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014911488:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762744.415667:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02f9c lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.415689:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.415695:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.415703:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02f9c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.415722:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02f9c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.420456:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02f9c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.420490:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02f9c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.420499:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02f9c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.420518:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf587600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762744.420524:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02f9c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.420534:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02f9c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.420552:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02f9c lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.420558:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02f9c lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762744.420619:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014911488:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5020us (98249us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762744.420628:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 223 00000100:00100000:0.0:1789762744.420638:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267719358922 time 267719462102 state 0x2 ref 0 00000100:00100000:0.0:1789762744.423813:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762744.423935:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014911616 00000100:00100000:3.0:1789762744.423961:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014911616 00000100:00100000:3.0:1789762744.424020:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267719358922 time 267719462102 state 0x0 ref 1 00000100:00100000:3.0:1789762744.424038:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267819462102 time 267719462102 state 0x1 ref 1 00000100:00100000:3.0:1789762744.515722:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267819462102 time 267819599263 state 0x0 ref 1 00000100:00100000:3.0:1789762744.515771:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014911616:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762744.515870:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02fa3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.515895:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762744.515902:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762744.515911:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02fa3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.515949:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02fa3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.520811:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02fa3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.520826:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02fa3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.520836:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02fa3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.520860:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762744.520866:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02fa3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.520876:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02fa3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.520893:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02fa3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.520901:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b02fa3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762744.520963:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014911616:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5213us (97150us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762744.520973:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 224 00000100:00100000:3.0:1789762744.520984:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267819462102 time 267819599263 state 0x2 ref 0 00000100:00100000:0.0:1789762744.524378:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014911744 00000100:00100000:0.0:1789762744.524389:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014911744 00000100:00100000:0.0:1789762744.524437:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267819462102 time 267819599263 state 0x0 ref 1 00000100:00100000:0.0:1789762744.524451:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267919599263 time 267819599263 state 0x1 ref 1 00000100:00100000:0.0:1789762744.615828:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267919599263 time 267919716159 state 0x0 ref 1 00000100:00100000:0.0:1789762744.615865:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014911744:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762744.615915:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02faa lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.615938:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.615945:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.615954:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02faa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.615975:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02faa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.620290:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02faa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.620307:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02faa lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.620344:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02faa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.620355:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf587600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762744.620361:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02faa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.620374:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02faa lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.620396:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02faa lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.620407:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02faa lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762744.620478:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014911744:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4626us (96205us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762744.620492:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 225 00000100:00100000:0.0:1789762744.620506:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267919599263 time 267919716159 state 0x2 ref 0 00000100:00100000:2.0:1789762744.622822:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762744.622872:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014911872 00000100:00100000:3.0:1789762744.622882:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014911872 00000100:00100000:3.0:1789762744.622926:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 267919599263 time 267919716159 state 0x0 ref 1 00000100:00100000:3.0:1789762744.622939:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268019716159 time 267919716159 state 0x1 ref 1 00000100:00100000:3.0:1789762744.715919:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268019716159 time 268019811071 state 0x0 ref 1 00000100:00100000:3.0:1789762744.715957:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014911872:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762744.716001:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fb1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.716018:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762744.716022:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762744.716028:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fb1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.716043:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fb1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.720657:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fb1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.720676:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fb1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.720689:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fb1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.720698:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762744.720704:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fb1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.720734:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fb1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.720759:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fb1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.720810:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fb1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762744.720917:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014911872:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4952us (98081us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762744.720936:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 226 00000100:00100000:3.0:1789762744.720953:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268019716159 time 268019811071 state 0x2 ref 0 00000100:00100000:3.0:1789762744.723387:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762744.723440:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014912000 00000100:00100000:0.0:1789762744.723450:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014912000 00000100:00100000:0.0:1789762744.723496:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268019716159 time 268019811071 state 0x0 ref 1 00000100:00100000:0.0:1789762744.723522:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268119811071 time 268019811071 state 0x1 ref 1 00000100:00100000:0.0:1789762744.815996:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268119811071 time 268119893022 state 0x0 ref 1 00000100:00100000:0.0:1789762744.816021:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014912000:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762744.816059:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02fb8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.816073:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.816077:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762744.816081:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02fb8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.816133:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02fb8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.818630:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02fb8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.818641:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02fb8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.818648:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02fb8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.818665:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebf587600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762744.818669:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02fb8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.818675:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02fb8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.818689:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02fb8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762744.818694:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf14032a/0x416193e012b02fb8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762744.818741:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014912000:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2730us (95354us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762744.818749:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 227 00000100:00100000:0.0:1789762744.818756:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268119811071 time 268119893022 state 0x2 ref 0 00000100:00100000:3.0:1789762744.821032:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014912128 00000100:00100000:3.0:1789762744.821040:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014912128 00000100:00100000:3.0:1789762744.821075:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268119811071 time 268119893022 state 0x0 ref 1 00000100:00100000:3.0:1789762744.821101:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268219893022 time 268119893022 state 0x1 ref 1 00000100:00100000:3.0:1789762744.916082:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268219893022 time 268219977847 state 0x0 ref 1 00000100:00100000:3.0:1789762744.916134:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014912128:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762744.916185:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fbf lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.916198:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762744.916202:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762744.916207:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fbf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.916218:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fbf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.919241:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fbf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.919252:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fbf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.919258:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fbf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.919263:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762744.919267:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fbf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.919273:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fbf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.919297:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fbf lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762744.919303:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fbf lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762744.919353:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014912128:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3230us (98361us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762744.919360:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 228 00000100:00100000:3.0:1789762744.919368:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268219893022 time 268219977847 state 0x2 ref 0 00000100:00100000:0.0:1789762744.921401:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014912256 00000100:00100000:0.0:1789762744.921412:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014912256 00000100:00100000:0.0:1789762744.921463:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268219893022 time 268219977847 state 0x0 ref 1 00000100:00100000:0.0:1789762744.921477:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268319977847 time 268219977847 state 0x1 ref 1 00000100:00100000:0.0:1789762745.016152:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268319977847 time 268320047115 state 0x0 ref 1 00000100:00100000:0.0:1789762745.016188:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014912256:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:2.0:1789762745.016214:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762745.016226:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754d34bf/0x416193e012b02fc6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.016240:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762745.016244:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762745.016248:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754d34bf/0x416193e012b02fc6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.016259:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754d34bf/0x416193e012b02fc6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.019462:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754d34bf/0x416193e012b02fc6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.019472:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754d34bf/0x416193e012b02fc6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.019480:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754d34bf/0x416193e012b02fc6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.019486:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762745.019490:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754d34bf/0x416193e012b02fc6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.019507:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754d34bf/0x416193e012b02fc6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.019521:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754d34bf/0x416193e012b02fc6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.019526:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754d34bf/0x416193e012b02fc6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762745.019577:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014912256:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3410us (98254us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762745.019584:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 229 00000100:00100000:2.0:1789762745.019592:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268319977847 time 268320047115 state 0x2 ref 0 00000100:00100000:3.0:1789762745.021501:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014912384 00000100:00100000:3.0:1789762745.021510:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014912384 00000100:00100000:3.0:1789762745.021545:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268319977847 time 268320047115 state 0x0 ref 1 00000100:00100000:3.0:1789762745.021556:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268420047115 time 268320047115 state 0x1 ref 1 00000100:00100000:3.0:1789762745.116228:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268420047115 time 268420123087 state 0x0 ref 1 00000100:00100000:3.0:1789762745.116250:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014912384:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762745.116286:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fcd lrc: 3/1,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.116303:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762745.116320:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762745.116328:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fcd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.116344:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fcd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.120368:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fcd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.120390:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fcd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.120401:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fcd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.120411:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762745.120417:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fcd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.120425:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fcd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.120441:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fcd lrc: 2/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762745.120447:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a025d9f9/0x416193e012b02fcd lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762745.120520:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014912384:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4273us (99047us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762745.120534:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 230 00000100:00100000:3.0:1789762745.120546:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268420047115 time 268420123087 state 0x2 ref 0 00000100:00100000:0.0:1789762745.146877:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762745.146931:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014912512 00000100:00100000:2.0:1789762745.146941:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014912512 00000100:00100000:2.0:1789762745.146998:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 11 00000100:00100000:2.0:1789762745.147008:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:5593:x1876702014912512:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762745.147021:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762745.147057:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fe209526/0x416193e012b02d11 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd376 expref: 6 pid: 10916 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.147078:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762745.147152:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:5593:x1876702014912512:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 146us (282us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762745.147169:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 11 00000100:00100000:0.0:1789762745.156154:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762745.156227:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014912640 00000100:00100000:2.0:1789762745.156236:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014912640 00000100:00100000:2.0:1789762745.156288:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 181 00000100:00100000:2.0:1789762745.156296:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:12244:x1876702014912640:12345-192.168.203.48@tcp:101:bash.0 00010000:00010000:2.0:1789762745.156313:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:0.0:1789762745.156326:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789762745.156328:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b02fd4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd37d expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.156379:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b02fdb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.156390:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b02fdb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.156400:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b02fdb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762745.156426:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1b:0x0]: returning size 16384 00000004:00200000:2.0:1789762745.156429:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762745.156443:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b02fdb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.156460:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b02fd4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd37d expref: 11 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.156473:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b02fdb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd37d expref: 10 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.156494:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1b:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762745.156496:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762745.156499:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbf800, rc 0) 00000100:00100000:2.0:1789762745.156557:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:12244:x1876702014912640:12345-192.168.203.48@tcp:101:bash.0 Request processed in 260us (408us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762745.156564:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 181 00000100:00100000:0.0:1789762745.160798:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014912896 00000100:00100000:0.0:1789762745.160803:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014912896 00000100:00100000:0.0:1789762745.160834:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 182 00000100:00100000:0.0:1789762745.160840:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:12568:x1876702014912896:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:0.0:1789762745.160847:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762745.160856:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184c00/0x416193e012b02fe2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd384 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762745.160910:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1b:0x0]: returning size 16384 00010000:00010000:0.0:1789762745.161079:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184c00/0x416193e012b02fe2 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd384 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.161110:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46000) 00010000:00010000:0.0:1789762745.161113:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762745.161116:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184c00/0x416193e012b02fe2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd384 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.161124:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86184c00, rc 0) 00000100:00100000:0.0:1789762745.161153:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:12568:x1876702014912896:12345-192.168.203.48@tcp:101:rm.0 Request processed in 314us (410us total) trans 4294967440 rc 301/301 00000100:00100000:0.0:1789762745.161159:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 182 00000100:00100000:0.0:1789762745.162682:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762745.162727:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014913024 00000100:00100000:3.0:1789762745.162732:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014913024 00000100:00100000:3.0:1789762745.162760:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 50 00000100:00100000:3.0:1789762745.162765:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:12568:x1876702014913024:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:0.0:1789762745.164542:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:12568:x1876702014913024:12345-192.168.203.48@tcp:37:rm.0 Request processed in 1775us (1859us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762745.164547:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 50 00000100:00100000:0.0:1789762745.165828:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014913280 00000100:00100000:0.0:1789762745.165833:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014913280 00000100:00100000:0.0:1789762745.165859:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 51 00000100:00100000:0.0:1789762745.165865:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:12568:x1876702014913280:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:0.0:1789762745.165958:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:12568:x1876702014913280:12345-192.168.203.48@tcp:35:rm.0 Request processed in 91us (169us total) trans 4294967441 rc 0/0 00000100:00100000:0.0:1789762745.165965:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 51 00000100:00100000:2.0:1789762745.166716:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762745.167814:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762745.167856:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014913664 00000100:00100000:2.0:1789762745.167862:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014913664 00000100:00100000:2.0:1789762745.167891:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 184 00000100:00100000:2.0:1789762745.167897:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:12568:x1876702014913664:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:2.0:1789762745.167927:0:5806:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7603224576 00000100:00100000:2.0:1789762745.167959:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:12568:x1876702014913664:12345-192.168.203.48@tcp:41:rm.0 Request processed in 62us (147us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762745.167966:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 184 00000100:00100000:3.0:1789762745.168819:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762745.168865:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014913920 00000100:00100000:0.0:1789762745.168874:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014913920 00000100:00100000:0.0:1789762745.168899:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 52 00000100:00100000:0.0:1789762745.168904:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b1a850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:12568:x1876702014913920:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:0.0:1789762745.169071:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b1a850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:12568:x1876702014913920:12345-192.168.203.48@tcp:35:rm.0 Request processed in 165us (253us total) trans 4294967443 rc 0/0 00000100:00100000:0.0:1789762745.169077:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 52 00000100:00100000:0.0:1789762745.169797:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762745.169863:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014914176 00000100:00100000:2.0:1789762745.169869:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014914176 00000100:00100000:2.0:1789762745.169903:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 185 00000100:00100000:2.0:1789762745.169913:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:12568:x1876702014914176:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:2.0:1789762745.169937:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762745.169949:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebf4f2e00/0x416193e012b02d03 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x1d:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd36f expref: 15 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.169961:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8812ca00/0x416193e012b02a16 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd368 expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.169985:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1d:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:2.0:1789762745.169997:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b02fdb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd37d expref: 13 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.170003:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1b:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762745.170006:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762745.170015:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b03005 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.170021:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b03005 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.170026:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b03005 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.170036:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b0300c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x1b:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.170041:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b0300c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1b:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.170045:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b0300c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1b:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.170290:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03013 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.170297:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03013 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x1d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.170308:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03013 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x1d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789762745.171098:0:5804:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].11, 64 off8896 00000004:00080000:2.0:1789762745.171109:0:5804:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 268475020941ns, next at 269475018306ns, rc = 0 00010000:00010000:2.0:1789762745.171156:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b0301a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171163:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b0301a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171169:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b0301a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171174:0:5804:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b0301a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x1d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171178:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b0301a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x1d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171182:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b0301a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1d:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171197:0:5804:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762745.171206:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03013 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x1d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171210:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03013 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x1d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171215:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03013 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x1d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171229:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762745.171234:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03013 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x1d:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171250:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03013 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x1d:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171260:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1d:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762745.171266:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762745.171269:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03013 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x1d:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171274:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03013 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x1d:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171298:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b03005 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171306:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b03005 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171311:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b03005 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171319:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762745.171324:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b03005 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171343:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b03005 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171369:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1b:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762745.171372:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762745.171375:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b03005 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.171383:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b03005 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762745.171392:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000016943513 reply state 000000007ad1998d transno 4294967444 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b0300c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1b:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762745.171397:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762745.171432:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:12568:x1876702014914176:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1522us (1635us total) trans 4294967444 rc 0/0 00000100:00100000:2.0:1789762745.171437:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 185 00010000:00010000:0.0:1789762745.171974:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b0300c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1b:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.171986:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b0300c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1b:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.171994:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b0300c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1b:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.172000:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbea00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762745.172004:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b0300c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1b:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.172012:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b0300c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x1b:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.172024:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1b:0x0].0x4009f790 (ffff981e82a26300) 00010000:00010000:0.0:1789762745.172026:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762745.172029:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b0300c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x1b:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762745.172034:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b0300c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x1b:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762745.172376:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762745.172406:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014914432 00000100:00100000:2.0:1789762745.172410:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014914432 00000100:00100000:2.0:1789762745.172434:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 53 00000100:00100000:2.0:1789762745.172439:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:12568:x1876702014914432:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:2.0:1789762745.172535:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:12568:x1876702014914432:12345-192.168.203.48@tcp:35:rm.0 Request processed in 93us (157us total) trans 4294967445 rc 0/0 00000100:00100000:2.0:1789762745.172542:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 53 00000100:00100000:3.0:1789762745.173203:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762745.174719:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03028 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.174728:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03028 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.174742:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03028 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.174746:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762745.174748:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03028 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.174752:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03028 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.174760:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179100) 00010000:00010000:2.0:1789762745.174762:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762745.174766:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03028 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762745.174788:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03028 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762746.065546:0:13746:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 269369457141ns, next at 269475018306ns, rc = 0 00000001:00080000:2.0:1789762746.074943:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967443 is committed 00000001:00080000:2.0:1789762746.074957:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967446 is committed 00000004:00080000:2.0:1789762746.074964:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 14 committed 00000004:00080000:2.0:1789762746.074977:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 13 committed 00000100:00100000:2.0:1789762746.075247:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022150528 00000100:00100000:2.0:1789762746.075257:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022150528 00000100:00080000:0.0:1789762746.075291:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762746.075324:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00000100:00100000:2.0:1789762746.075334:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022150528:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1789762746.075357:0:6647:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:11 count 1 00002000:00010000:2.0:1789762746.075385:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xb:0x0] initial LVB size: 104857600, mtime: 0x6aad9caf, atime: 0x0, ctime: 0x6aad9caf, blocks: 0x3200d 00010000:00010000:2.0:1789762746.075397:0:6647:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b03036 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762746.075411:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=269s v=(0 0) 00010000:00010000:2.0:1789762746.075414:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762746.075419:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b03036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762746.075432:0:6647:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b03036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762746.075439:0:6647:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b03036 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762746.075445:0:6647:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b03036 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762746.075459:0:6647:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762746.075486:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762746.075496:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b03036 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.075509:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbee00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762746.075514:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b03036 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.075526:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b03036 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.075548:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b03036 lrc: 1/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.075564:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b03036 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762746.075583:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762746.075585:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762746.075587:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762746.075898:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022150528:12345-0@lo:6:osp-syn-0-0.0 Request processed in 566us (674us total) trans 4294967673 rc 0/0 00000100:00100000:2.0:1789762746.075904:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000004:00080000:2.0:1789762746.076075:0:13746:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 269379987467ns, next at 255943130658ns, rc = 0 00000004:00080000:3.0:1789762746.076884:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 12 committed 00010000:00010000:0.0:1789762746.295186:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762746.466341:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762746.466374:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762746.466403:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762746.466417:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014914944 00000100:00100000:2.0:1789762746.466425:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014914944 00000100:00100000:0.0:1789762746.466436:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762746.466484:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 183 00000100:00100000:0.0:1789762746.466485:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762746.466501:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81380a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014914944:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762746.466514:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014915200 00000100:00100000:0.0:1789762746.466518:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014915200 00000100:00100000:0.0:1789762746.466559:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014915072 00000100:00100000:0.0:1789762746.466562:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014915072 00000100:00100000:2.0:1789762746.466587:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81380a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014914944:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 87us (208us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762746.466591:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 90 00000100:00100000:0.0:1789762746.466606:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014915200:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762746.466614:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 183 00000100:00100000:2.0:1789762746.466641:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014915328 00000100:00100000:2.0:1789762746.466645:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014915328 00000100:00100000:0.0:1789762746.466646:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014915200:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 45us (241us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762746.466655:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 90 00000100:00100000:0.0:1789762746.466676:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 91 00000100:00100000:0.0:1789762746.466690:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014915072:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762746.466690:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 184 00000100:00100000:2.0:1789762746.466706:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702014915328:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762746.466722:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014915072:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 37us (286us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762746.466730:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 91 00000100:00100000:2.0:1789762746.466745:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702014915328:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 45us (266us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762746.466751:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 184 00000100:00100000:0.0:1789762746.487244:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad03480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022150656:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762746.487298:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762746.487334:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022150656 00000100:00100000:2.0:1789762746.487341:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022150656 00000100:00100000:2.0:1789762746.487380:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 79 00000100:00100000:2.0:1789762746.487388:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022150656:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762746.487414:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762746.487456:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022150656:12345-0@lo:13:osp-pre-0-0.0 Request processed in 69us (159us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762746.487462:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 79 00000100:00100000:0.0:1789762746.487496:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad03480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022150656:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762746.999248:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022151168:0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.999267:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac6f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022150784:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762746.999279:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022151040:0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.999316:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762746.999332:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762746.999338:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762746.999379:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad02a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022151552:0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.999380:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad00700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022150912:0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.999409:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022150784 00000100:00100000:3.0:1789762746.999410:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762746.999416:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022150784 00000100:00100000:2.0:1789762746.999434:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762746.999462:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022150912 00000100:00100000:3.0:1789762746.999468:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022150912 00000100:00100000:0.0:1789762746.999477:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 80 00000100:00100000:2.0:1789762746.999487:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad03800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022151424:0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.999488:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022150784:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762746.999513:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00002000:00000024:0.0:1789762746.999523:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762746.999526:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850db050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022150912:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.999534:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762746.999589:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022151296 00000100:00100000:2.0:1789762746.999593:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022151296 00000100:00100000:0.0:1789762746.999598:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022150784:12345-0@lo:13:osp-pre-1-0.0 Request processed in 113us (267us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762746.999605:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850db050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022150912:12345-0@lo:400:kworker.0 Request processed in 82us (171us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762746.999619:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 80 00000100:00100000:2.0:1789762746.999620:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022151552 00000100:00100000:3.0:1789762746.999625:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00100000:2.0:1789762746.999635:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022151552 00000100:00100000:0.0:1789762746.999664:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:2.0:1789762746.999671:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 92 00000100:00100000:0.0:1789762746.999677:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022151040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.999683:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 93 00000100:00100000:2.0:1789762746.999684:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022151296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.999705:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022151552:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762746.999719:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022151040:12345-0@lo:400:kworker.0 Request processed in 47us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762746.999726:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:2.0:1789762746.999749:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022151296:12345-0@lo:400:kworker.0 Request processed in 69us (404us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762746.999771:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 92 00000100:00100000:3.0:1789762746.999777:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022151552:12345-0@lo:400:kworker.0 Request processed in 75us (364us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762746.999790:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022151168:0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.999791:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 93 00000100:00100000:2.0:1789762746.999833:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad00380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022151296:0@lo:400:kworker.0 00000100:00100000:3.0:1789762746.999845:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022151040:0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.999879:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac6f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022150784:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762746.999883:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad02a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022151552:0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.999898:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad00700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022150912:0@lo:400:kworker.0 00000100:00100000:2.0:1789762746.999908:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad03800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022151424:0@lo:400:kworker.0 00010000:00010000:3.0:1789762749.367244:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762750.391233:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762750.600795:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967673 is committed 00000100:00100000:3.0:1789762751.586258:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762751.586292:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762751.586319:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762751.586323:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014915712 00000100:00100000:0.0:1789762751.586330:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014915712 00000100:00100000:2.0:1789762751.586359:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014915584 00000100:00080000:3.0:1789762751.586368:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762751.586375:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014915584 00000100:00100000:0.0:1789762751.586389:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 94 00000100:00100000:0.0:1789762751.586403:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014915712:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:3.0:1789762751.586413:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762751.586434:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 188 00000100:00100000:2.0:1789762751.586444:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014915584:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762751.586470:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014915712:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 71us (174us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762751.586484:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014915584:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 41us (164us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762751.586489:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 94 00000100:00100000:2.0:1789762751.586500:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 188 00000100:00100000:2.0:1789762751.588454:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014915840 00000100:00100000:2.0:1789762751.588463:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014915840 00000100:00100000:2.0:1789762751.588513:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 123 00000100:00100000:2.0:1789762751.588521:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2362:x1876702014915840:12345-192.168.203.48@tcp:9:kworker.0 00000020:00000080:2.0:1789762751.588538:0:6648:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x416193e012b02802 00000100:00080000:2.0:1789762751.588545:0:6648:0:(import.c:65:import_set_state_nolock()) ffff981ebc4ce000 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1789762751.588578:0:6648:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967362 00000100:00100000:3.0:1789762751.588586:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014915968 00000100:00100000:3.0:1789762751.588594:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014915968 00000100:00100000:3.0:1789762751.588647:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 189 00000100:00100000:3.0:1789762751.588659:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014915968:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762751.588706:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014915968:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 50us (244us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762751.588714:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 189 00000100:00100000:2.0:1789762751.588808:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+2:2362:x1876702014915840:12345-192.168.203.48@tcp:9:kworker.0 Request processed in 286us (397us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762751.588818:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 123 00000100:00100000:0.0:1789762751.607238:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad00e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022151680:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762751.607281:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762751.607316:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022151680 00000100:00100000:2.0:1789762751.607322:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022151680 00000100:00100000:2.0:1789762751.607368:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 81 00000100:00100000:2.0:1789762751.607383:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022151680:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762751.607408:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762751.607446:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022151680:12345-0@lo:13:osp-pre-0-0.0 Request processed in 71us (164us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762751.607452:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 81 00000100:00100000:0.0:1789762751.607474:0:3273:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967673 gen 2 00000004:00080000:0.0:1789762751.607482:0:3273:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebf47dc00, transno 4294967673 00000100:00100000:0.0:1789762751.607505:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad00e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022151680:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762752.119248:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebf47f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022152064:0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.119258:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac6d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022151808:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762752.119307:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebf47ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022152192:0@lo:400:kworker.0 00000100:00100000:0.0:1789762752.119341:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762752.119345:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762752.119376:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022152064 00000100:00100000:3.0:1789762752.119384:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022152064 00000100:00100000:0.0:1789762752.119421:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebf47e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022152576:0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.119432:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebf47ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022152320:0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.119435:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:3.0:1789762752.119448:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022152064:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762752.119452:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762752.119486:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762752.119500:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022151808 00000100:00100000:0.0:1789762752.119505:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022151808 00000100:00100000:3.0:1789762752.119529:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022152064:12345-0@lo:400:kworker.0 Request processed in 81us (192us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762752.119540:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00100000:0.0:1789762752.119552:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 82 00000100:00100000:0.0:1789762752.119563:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022151808:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762752.119569:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022152576 00000100:00100000:3.0:1789762752.119569:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022152448 00000100:00100000:3.0:1789762752.119575:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022152448 00000100:00100000:2.0:1789762752.119577:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022152576 00002000:00000024:0.0:1789762752.119590:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762752.119614:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:3.0:1789762752.119626:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022152448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.119632:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022152320 00000100:00100000:2.0:1789762752.119638:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022152320 00000100:00100000:0.0:1789762752.119647:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022151808:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762752.119658:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 82 00000100:00100000:3.0:1789762752.119693:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022152448:12345-0@lo:400:kworker.0 Request processed in 68us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762752.119697:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 95 00000100:00100000:0.0:1789762752.119710:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 96 00000100:00100000:3.0:1789762752.119711:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:2.0:1789762752.119729:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022152576:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762752.119743:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022152320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.119792:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebf47f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022152064:0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.119797:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022152576:12345-0@lo:400:kworker.0 Request processed in 83us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762752.119801:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022152320:12345-0@lo:400:kworker.0 Request processed in 63us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762752.119815:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 95 00000100:00100000:0.0:1789762752.119825:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 96 00000100:00100000:3.0:1789762752.119829:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebf47e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022152576:0@lo:400:kworker.0 00000100:00100000:2.0:1789762752.119862:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762752.119896:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022152192 00000100:00100000:3.0:1789762752.119901:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022152192 00000100:00100000:3.0:1789762752.119936:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1789762752.119945:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022152192:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762752.119980:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022152192:12345-0@lo:400:kworker.0 Request processed in 38us (121us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762752.119985:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:2.0:1789762752.120010:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebf47ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022152192:0@lo:400:kworker.0 00010000:00010000:3.0:1789762752.439229:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762753.463256:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:2.0:1789762755.200179:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:2.0:1789762755.200193:0:7605:0:(genops.c:885:class_export_put()) final put ffff981ebc4cd000/f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:3.0:1789762755.200221:0:37:0:(genops.c:844:class_export_destroy()) destroying export ffff981ebc4cd000/f1bd28de-ee78-4d18-bad3-fa7cec968893 for lustre-OST0001 00000020:00000080:3.0:1789762755.200275:0:37:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff981ebc4ce000 for lustre-OST0001 00010000:00010000:3.0:1789762756.535224:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762756.706338:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762756.706375:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762756.706397:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762756.706445:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762756.706456:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014916096 00000100:00100000:2.0:1789762756.706463:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014916096 00000100:00100000:0.0:1789762756.706469:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014916480 00000100:00100000:0.0:1789762756.706475:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014916480 00000100:00100000:3.0:1789762756.706488:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014916352 00000100:00100000:3.0:1789762756.706493:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014916352 00000100:00100000:2.0:1789762756.706508:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 282 00000100:00100000:0.0:1789762756.706518:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 194 00000100:00100000:2.0:1789762756.706521:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702014916096:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762756.706533:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 97 00000100:00100000:0.0:1789762756.706541:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014916480:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762756.706546:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014916352:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762756.706579:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702014916096:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 61us (200us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762756.706588:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014916480:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 49us (187us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762756.706595:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 282 00000100:00100000:3.0:1789762756.706597:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014916352:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 54us (156us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762756.706617:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 194 00000100:00100000:3.0:1789762756.706623:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 97 00000100:00100000:2.0:1789762756.727244:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad02300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022152704:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762756.727283:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762756.727324:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022152704 00000100:00100000:0.0:1789762756.727331:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022152704 00000100:00100000:0.0:1789762756.727376:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 83 00000100:00100000:0.0:1789762756.727384:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022152704:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762756.727408:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762756.727447:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022152704:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (163us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762756.727454:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 83 00000100:00100000:2.0:1789762756.727482:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad02300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022152704:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762757.239255:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022152960:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762757.239277:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac6d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022153216:0@lo:400:kworker.0 00000100:00100000:3.0:1789762757.239281:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac6df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022153088:0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.239324:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762757.239338:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac6c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022152832:0@lo:400:kworker.0 00000100:00100000:0.0:1789762757.239350:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762757.239380:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762757.239382:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac6ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022153472:0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.239417:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762757.239423:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac6c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022153344:0@lo:400:kworker.0 00000100:00100000:0.0:1789762757.239428:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022152832 00000100:00100000:0.0:1789762757.239435:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022152832 00000100:00100000:3.0:1789762757.239451:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762757.239461:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022153472 00000100:00100000:2.0:1789762757.239466:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022153472 00000100:00100000:0.0:1789762757.239480:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283 00000100:00100000:0.0:1789762757.239494:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022152832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762757.239500:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762757.239509:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 98 00000100:00100000:2.0:1789762757.239523:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022153472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762757.239554:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022152832:12345-0@lo:400:kworker.0 Request processed in 62us (174us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762757.239555:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022153216 00000100:00100000:3.0:1789762757.239561:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022153216 00000100:00100000:0.0:1789762757.239565:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283 00000100:00100000:2.0:1789762757.239584:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022153472:12345-0@lo:400:kworker.0 Request processed in 64us (167us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762757.239591:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 98 00000100:00100000:3.0:1789762757.239610:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022153344 00000100:00100000:0.0:1789762757.239614:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022153088 00000100:00100000:3.0:1789762757.239616:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022153344 00000100:00100000:0.0:1789762757.239619:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022153088 00000100:00100000:2.0:1789762757.239641:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022152960:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762757.239654:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:2.0:1789762757.239658:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac6ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022153472:0@lo:400:kworker.0 00000100:00100000:3.0:1789762757.239664:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:0.0:1789762757.239665:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022153216:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762757.239699:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022153088:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762757.239723:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022153216:12345-0@lo:400:kworker.0 Request processed in 60us (385us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762757.239736:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00100000:3.0:1789762757.239757:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022153088:12345-0@lo:400:kworker.0 Request processed in 81us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762757.239766:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac6c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022152832:0@lo:400:kworker.0 00000100:00100000:3.0:1789762757.239767:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:0.0:1789762757.239789:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac6d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022153216:0@lo:400:kworker.0 00000100:00100000:2.0:1789762757.239791:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac6c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022153344:0@lo:400:kworker.0 00000100:00100000:3.0:1789762757.239807:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac6df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022153088:0@lo:400:kworker.0 00010000:00010000:0.0:1789762757.559253:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762760.631275:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762761.655267:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762761.826449:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762761.826494:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762761.826525:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762761.826580:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762761.826600:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014916608 00000100:00100000:2.0:1789762761.826607:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014916736 00000100:00100000:0.0:1789762761.826618:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014916608 00000100:00100000:2.0:1789762761.826619:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014916736 00000100:00100000:3.0:1789762761.826641:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014916992 00000100:00100000:3.0:1789762761.826678:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014916992 00000100:00100000:0.0:1789762761.826760:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 284 00000100:00100000:0.0:1789762761.826786:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702014916608:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762761.826801:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 198 00000100:00100000:3.0:1789762761.826821:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 199 00000100:00100000:2.0:1789762761.826841:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014916736:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762761.826846:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014916992:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762761.826918:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702014916608:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 140us (478us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762761.826930:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 284 00000100:00100000:2.0:1789762761.826966:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014916736:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 140us (463us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762761.826986:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 198 00000100:00100000:3.0:1789762761.826986:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014916992:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 141us (399us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762761.827017:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 199 00000100:00100000:2.0:1789762761.847382:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762761.847412:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022153600:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762761.847501:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762761.847625:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022153600 00000100:00100000:0.0:1789762761.847639:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022153600 00000100:00100000:0.0:1789762761.847776:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 85 00000100:00100000:0.0:1789762761.847791:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022153600:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762761.847832:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 241893376 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762761.847905:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022153600:12345-0@lo:13:osp-pre-0-0.0 Request processed in 115us (404us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762761.847921:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 85 00000100:00100000:2.0:1789762761.848003:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022153600:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762762.359229:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc435180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022153728:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762762.359292:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762762.359343:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022153728 00000100:00100000:2.0:1789762762.359353:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022153728 00000100:00080000:3.0:1789762762.359403:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762762.359436:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 86 00000100:00100000:2.0:1789762762.359453:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022153728:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789762762.359493:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762762.359576:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022153728:12345-0@lo:13:osp-pre-1-0.0 Request processed in 130us (284us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762762.359585:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 86 00000100:00100000:0.0:1789762762.359609:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc435180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022153728:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762762.360208:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac6f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022153856:0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.360252:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762762.360278:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022153856 00000100:00100000:0.0:1789762762.360286:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022153856 00000100:00100000:3.0:1789762762.360299:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789762762.360308:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac6c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022154240:0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.360335:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:3.0:1789762762.360346:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762762.360354:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022153856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.360397:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022154240 00000100:00100000:3.0:1789762762.360402:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022154240 00000100:00100000:0.0:1789762762.360415:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022153856:12345-0@lo:400:kworker.0 Request processed in 62us (160us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762762.360425:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:3.0:1789762762.360447:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 201 00000100:00100000:3.0:1789762762.360456:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022154240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.360501:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022154240:12345-0@lo:400:kworker.0 Request processed in 46us (158us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762762.360509:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 201 00000100:00100000:3.0:1789762762.360559:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac6c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022154240:0@lo:400:kworker.0 00000100:00100000:3.0:1789762762.360574:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac6f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022153856:0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.360602:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022154112 00000100:00100000:0.0:1789762762.360606:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022154112 00000100:00080000:3.0:1789762762.360635:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762762.360661:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:0.0:1789762762.360668:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022154112:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762762.360703:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022154112:12345-0@lo:400:kworker.0 Request processed in 36us (145us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762762.360709:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00010000:00010000:0.0:1789762762.680356:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762763.152588:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.48@tcp 00000100:00100000:0.0:1789762763.152610:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:0.0:1789762763.152618:0:14206:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:0.0:1789762763.152624:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.48@tcp 00000100:00100000:0.0:1789762763.152631:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:0.0:1789762763.152648:0:14206:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981d81a45700 rate 20 gen 0 00000100:00100000:0.0:1789762763.152651:0:14206:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:0.0:1789762763.152656:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.48@tcp 00000100:00100000:0.0:1789762763.152663:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:0.0:1789762763.152666:0:14206:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981d81a45a00 rate 20 gen 0 00000100:00100000:0.0:1789762763.152746:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.48@tcp 00000100:00100000:0.0:1789762763.152754:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:0.0:1789762763.152758:0:14206:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:0.0:1789762763.152762:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.48@tcp 00000100:00100000:0.0:1789762763.152769:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:0.0:1789762763.152773:0:14206:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981d81a45000 rate 10 gen 0 00000100:00100000:0.0:1789762763.152781:0:14206:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:0.0:1789762763.152784:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.48@tcp 00000100:00100000:0.0:1789762763.152792:0:14206:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:0.0:1789762763.152796:0:14206:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981d81a45500 rate 10 gen 0 00000100:00100000:3.0:1789762763.162423:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.162474:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014917120 00000100:00100000:2.0:1789762763.162488:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014917120 00000100:00080000:3.0:1789762763.162532:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762763.162579:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 187 00000100:00100000:2.0:1789762763.162595:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc557b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:13021:x1876702014917120:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:2.0:1789762763.162625:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762763.162665:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b0303d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd392 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.162738:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03044 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.162753:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03044 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.162765:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03044 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762763.162804:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762763.162807:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762763.162825:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03044 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.162851:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b0303d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd392 expref: 7 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.162868:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03044 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd392 expref: 6 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.162875:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26600) 00010000:00010000:2.0:1789762763.162878:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762763.162882:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbf000, rc 0) 00000100:00100000:2.0:1789762763.162952:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc557b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:13021:x1876702014917120:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 364us (534us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762763.162958:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 187 00000100:00100000:0.0:1789762763.163935:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762763.166446:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03052 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.166454:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03052 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.166459:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03052 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.166463:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762763.166466:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03052 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.166470:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03052 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.166477:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179400) 00010000:00010000:0.0:1789762763.166479:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762763.166481:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03052 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.166485:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03052 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762763.170301:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.171441:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.171485:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014917760 00000100:00100000:3.0:1789762763.171491:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014917760 00000100:00100000:3.0:1789762763.171526:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 190 00000100:00100000:3.0:1789762763.171535:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:13022:x1876702014917760:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762763.171551:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762763.171552:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1789762763.171564:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b03060 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd399 expref: 7 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171575:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179600) 00010000:00010000:3.0:1789762763.171580:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762763.171587:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b03067 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd3a0 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171680:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031a00/0x416193e012b0306e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171687:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031a00/0x416193e012b0306e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171694:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031a00/0x416193e012b0306e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171704:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03075 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171709:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03075 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171713:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03075 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171762:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0307c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171777:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0307c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171782:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0307c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762763.171788:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1e:0x0]: returning size 16384 00010000:00010000:3.0:1789762763.171797:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0307c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171801:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0307c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171805:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0307c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171809:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762763.171812:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0307c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171816:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0307c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171823:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x0 (ffff981e83a46200) 00010000:00010000:3.0:1789762763.171825:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762763.171827:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0307c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171831:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0307c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171843:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031a00/0x416193e012b0306e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171854:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031a00/0x416193e012b0306e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171859:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031a00/0x416193e012b0306e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171862:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762763.171864:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031a00/0x416193e012b0306e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171869:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031a00/0x416193e012b0306e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171875:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46000) 00010000:00010000:3.0:1789762763.171876:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762763.171878:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031a00/0x416193e012b0306e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171882:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031a00/0x416193e012b0306e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171889:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03075 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171894:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03075 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171898:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03075 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171901:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762763.171909:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03075 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171914:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03075 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171919:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e83a46600) 00010000:00010000:3.0:1789762763.171921:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762763.171923:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03075 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171926:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03075 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171945:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b03067 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3a0 expref: 7 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171951:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46000) 00010000:00010000:3.0:1789762763.171952:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762763.171955:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b03067 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3a0 expref: 7 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.171962:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e80031000, rc 0) 00000100:00100000:3.0:1789762763.171996:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:13022:x1876702014917760:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 463us (558us total) trans 0 rc 301/301 00000100:00100000:3.0:1789762763.172001:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 190 00000100:00100000:3.0:1789762763.172946:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.174342:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.175761:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.175790:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014918400 00000100:00100000:3.0:1789762763.175795:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014918400 00000100:00100000:3.0:1789762763.175829:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 193 00000100:00100000:3.0:1789762763.175853:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:13022:x1876702014918400:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:3.0:1789762763.175899:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b030ad lrc: 3/0,1 mode: --/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.175911:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b030ad lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.175922:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b030ad lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762763.176097:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f1cb6e02 reply state 00000000ae15b536 transno 4294967449 ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b030ad lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762763.176108:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762763.176145:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1e:0x0]: returning size 16384 00000100:00100000:3.0:1789762763.176205:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13022:x1876702014918400:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 358us (442us total) trans 4294967449 rc 0/0 00000100:00100000:3.0:1789762763.176212:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 193 00010000:00010000:2.0:1789762763.176692:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b030ad lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.176702:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b030ad lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.176707:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b030ad lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.176712:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762763.176715:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b030ad lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.176719:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b030ad lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.176729:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:2.0:1789762763.176731:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789762763.176734:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b030ad lrc: 2/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.176738:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b030ad lrc: 0/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.176926:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.176977:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014918656 00000100:00100000:0.0:1789762763.176983:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014918656 00000100:00100000:0.0:1789762763.177027:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 54 00000100:00100000:0.0:1789762763.177035:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:13022:x1876702014918656:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:0.0:1789762763.177237:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13022:x1876702014918656:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 203us (312us total) trans 4294967450 rc 0/0 00000100:00100000:0.0:1789762763.177245:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 54 00000100:00100000:3.0:1789762763.181272:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.182615:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:3.0:1789762763.183548:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b030d0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.183556:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b030d0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.183561:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b030d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.183581:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762763.183585:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b030d0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.183600:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b030d0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.183609:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x0 (ffff981ec2179700) 00010000:00010000:3.0:1789762763.183612:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762763.183615:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b030d0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.183619:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b030d0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.194538:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.194582:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014919168 00000100:00100000:0.0:1789762763.194589:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014919168 00000100:00100000:0.0:1789762763.194646:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 196 00000100:00100000:0.0:1789762763.194658:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86006d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13027:x1876702014919168:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762763.194689:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762763.194701:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185800/0x416193e012b030d7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd3bc expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194757:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b030de lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194766:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b030de lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194773:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b030de lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194786:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b030e5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194791:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b030e5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194797:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b030e5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194872:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b030ec lrc: 3/1,0 mode: --/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194878:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b030ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194885:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b030ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762763.194902:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1e:0x0]: returning size 16384 00000004:00200000:0.0:1789762763.194904:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762763.194915:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b030ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194922:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b030de lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194926:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b030de lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194930:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b030de lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194934:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762763.194937:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b030de lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194942:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b030de lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194951:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179000) 00010000:00010000:0.0:1789762763.194959:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762763.194962:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b030de lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194967:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b030de lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194976:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b030e5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194981:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b030e5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194985:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b030e5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194990:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762763.194992:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b030e5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.194996:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b030e5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.195004:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3d00) 00010000:00010000:0.0:1789762763.195006:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762763.195008:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b030e5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.195011:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b030e5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.195034:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185800/0x416193e012b030d7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3bc expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.195057:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b030ec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3bc expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.195063:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x0 (ffff981ea0aa3300) 00010000:00010000:0.0:1789762763.195065:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789762763.195067:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86185400, rc 0) 00000100:00100000:0.0:1789762763.195133:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86006d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:13027:x1876702014919168:12345-192.168.203.48@tcp:101:dd.0 Request processed in 477us (601us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762763.195140:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 196 00000100:00100000:0.0:1789762763.196174:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.196207:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014919424 00000100:00100000:2.0:1789762763.196215:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014919424 00000100:00100000:2.0:1789762763.196269:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 197 00000100:00100000:2.0:1789762763.196281:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:13027:x1876702014919424:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762763.196296:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762763.196309:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfa00/0x416193e012b030f3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd3c3 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.196415:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbec00/0x416193e012b030fa lrc: 3/0,1 mode: --/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.196426:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbec00/0x416193e012b030fa lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.196435:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbec00/0x416193e012b030fa lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.197610:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe400/0x416193e012b03101 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.197623:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe400/0x416193e012b03101 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.197630:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe400/0x416193e012b03101 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762763.198126:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xc:0x0], index 0: 0 00000004:00080000:2.0:1789762763.198138:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x8:0x0], index 1: 0 00010000:00010000:2.0:1789762763.198258:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbee00/0x416193e012b03108 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.198274:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbee00/0x416193e012b03108 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.198297:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbee00/0x416193e012b03108 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.198331:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbec00/0x416193e012b030fa lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.198337:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbec00/0x416193e012b030fa lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.198342:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbec00/0x416193e012b030fa lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.198345:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbec00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762763.198348:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbec00/0x416193e012b030fa lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.198352:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbec00/0x416193e012b030fa lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.198359:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x0 (ffff981ea0aa3300) 00010000:00010000:2.0:1789762763.198361:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762763.198364:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbec00/0x416193e012b030fa lrc: 2/0,0 mode: --/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.198368:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbec00/0x416193e012b030fa lrc: 0/0,0 mode: --/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762763.198377:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000aedcbd91 reply state 0000000056ccbd7b transno 4294967452 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe400/0x416193e012b03101 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.198382:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762763.198410:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfa00/0x416193e012b030f3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3c3 expref: 15 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.198420:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbee00/0x416193e012b03108 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3c3 expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.198425:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff981e82a26500) 00010000:00010000:2.0:1789762763.198427:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762763.198428:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbee00, rc 0) 00000100:00100000:2.0:1789762763.198463:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:13027:x1876702014919424:12345-192.168.203.48@tcp:101:dd.0 Request processed in 2187us (2290us total) trans 4294967452 rc 0/0 00000100:00100000:2.0:1789762763.198468:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 197 00010000:00010000:3.0:1789762763.199010:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe400/0x416193e012b03101 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.199018:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe400/0x416193e012b03101 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.199023:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe400/0x416193e012b03101 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.199027:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762763.199029:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe400/0x416193e012b03101 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.199034:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe400/0x416193e012b03101 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.199042:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x77289514 (ffff981e82a26d00) 00010000:00010000:3.0:1789762763.199044:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762763.199046:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe400/0x416193e012b03101 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.199056:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe400/0x416193e012b03101 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.201009:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.201051:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014919552 00000100:00100000:0.0:1789762763.201057:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014919552 00000100:00080000:2.0:1789762763.201122:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762763.201133:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 124 00000100:00100000:0.0:1789762763.201145:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86007800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:13027:x1876702014919552:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762763.201154:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762763.201167:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b03116 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd3ca expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762763.201187:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xc:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789762763.201200:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=286s v=(0 0) 00010000:00010000:0.0:1789762763.201204:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762763.201211:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b03116 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3ca expref: 6 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.201223:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b03116 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3ca expref: 6 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.201240:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86184000, rc 0) 00000100:00100000:0.0:1789762763.201274:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86007800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:13027:x1876702014919552:12345-192.168.203.48@tcp:101:dd.0 Request processed in 133us (267us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762763.201281:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 124 00000100:00100000:2.0:1789762763.208131:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.215164:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.215196:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014919808 00000100:00100000:3.0:1789762763.215202:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014919808 00002000:00100000:3.0:1789762763.215222:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xc:0x0] (1048576->2097151) req@ffff981e86006680 x1876702014919808/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762763.215230:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e86006680 with x1876702014919808 ext(1048576->2097151) 00002000:00010000:3.0:1789762763.215233:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e86006680 00000100:00100000:3.0:1789762763.215241:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 2 deadline 286562159665 time 286512200429 state 0x0 ref 1 00000100:00100000:3.0:1789762763.215249:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 2 deadline 286562200429 time 286512200429 state 0x1 ref 1 00000100:00100000:3.0:1789762763.215268:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 1 deadline 286562200429 time 286519177742 state 0x0 ref 1 00000100:00100000:3.0:1789762763.215276:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014919808:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762763.217767:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014919808:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2490us (2602us total) trans 4294967675 rc 0/0 00000100:00100000:3.0:1789762763.217774:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 232 00000100:00100000:3.0:1789762763.217781:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 1 deadline 286562200429 time 286519177742 state 0x2 ref 0 00002000:00100000:3.0:1789762763.217789:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xc:0x0] (1048576->2097151) req@ffff981e86006680 x1876702014919808/t4294967675(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762763.217807:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e86006680 with x1876702014919808 ext(1048576->2097151) 00002000:00010000:3.0:1789762763.217810:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e86006680 00000100:00100000:2.0:1789762763.221659:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.221691:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014919936 00000100:00100000:3.0:1789762763.221697:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014919936 00002000:00100000:3.0:1789762763.221717:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xc:0x0] (2097152->3145727) req@ffff981e86005180 x1876702014919936/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762763.221725:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e86005180 with x1876702014919936 ext(2097152->3145727) 00002000:00010000:3.0:1789762763.221728:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e86005180 00000100:00100000:3.0:1789762763.221736:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 1 deadline 286562200429 time 286519177742 state 0x0 ref 1 00000100:00100000:3.0:1789762763.221744:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 1 deadline 286569177742 time 286519177742 state 0x1 ref 1 00000100:00100000:3.0:1789762763.221764:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286569177742 time 286525674038 state 0x0 ref 1 00000100:00100000:3.0:1789762763.221772:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2365:x1876702014919936:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762763.229475:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.229557:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014920064 00000100:00100000:0.0:1789762763.229563:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014920064 00002000:00100000:0.0:1789762763.229585:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0xc:0x0] (3145728->4194303) req@ffff981e86007100 x1876702014920064/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762763.229592:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e86007100 with x1876702014920064 ext(3145728->4194303) 00002000:00010000:0.0:1789762763.229596:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e86007100 00000100:00100000:0.0:1789762763.229604:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286569177742 time 286525674038 state 0x0 ref 1 00000100:00100000:0.0:1789762763.229612:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286575674038 time 286525674038 state 0x1 ref 1 00000100:00100000:3.0:1789762763.229802:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.229851:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014920192 00000100:00100000:2.0:1789762763.229857:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014920192 00000100:00100000:2.0:1789762763.229888:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 125 00000100:00100000:2.0:1789762763.229894:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0+-99:2362:x1876702014920192:12345-192.168.203.48@tcp:8:dd.0 00010000:00080000:2.0:1789762763.229915:0:6881:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp t0 exp 0000000000000000 cur 286 last 0 00000020:00000080:2.0:1789762763.229936:0:6881:0:(genops.c:1385:class_connect()) connect: client f1bd28de-ee78-4d18-bad3-fa7cec968893, cookie 0x416193e012b0311d 00002000:00100000:2.0:1789762763.229945:0:6881:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli f1bd28de-ee78-4d18-bad3-fa7cec968893/ffff981ebf589800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:2.0:1789762763.229951:0:6881:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:2.0:1789762763.230081:0:6881:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:2.0:1789762763.230133:0:6881:0:(import.c:65:import_set_state_nolock()) ffff981ebf588800 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789762763.230172:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2362:x1876702014920192:12345-192.168.203.48@tcp:8:dd.0 Request processed in 277us (371us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762763.230178:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 125 00000100:00100000:2.0:1789762763.230887:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762763.230991:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762763.238333:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.238383:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014920448 00000100:00100000:2.0:1789762763.238389:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014920448 00002000:00100000:2.0:1789762763.238408:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0x8:0x0] (0->1048575) req@ffff981ebac6f100 x1876702014920448/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.238416:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac6f100 with x1876702014920448 ext(0->1048575) 00002000:00010000:2.0:1789762763.238420:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac6f100 00000100:00100000:2.0:1789762763.238429:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286575674038 time 286525674038 state 0x1 ref 2 00000100:00100000:2.0:1789762763.245043:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.245106:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014920576 00000100:00100000:0.0:1789762763.245113:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014920576 00002000:00100000:0.0:1789762763.245138:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x8:0x0] (1048576->2097151) req@ffff981e86004a80 x1876702014920576/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762763.245150:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e86004a80 with x1876702014920576 ext(1048576->2097151) 00002000:00010000:0.0:1789762763.245154:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e86004a80 00000100:00100000:0.0:1789762763.245166:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286575674038 time 286525674038 state 0x1 ref 3 00000100:00100000:0.0:1789762763.271817:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286625717523 time 286575717523 state 0x1 ref 3 00000100:00100000:0.0:1789762763.271840:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2366:x1876702014920064:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762763.274479:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:2366:x1876702014920064:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2639us (45003us total) trans 4294967677 rc 0/0 00000100:00100000:2.0:1789762763.274487:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 234 00002000:00100000:2.0:1789762763.274495:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0xc:0x0] (3145728->4194303) req@ffff981e86007100 x1876702014920064/t4294967677(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.274503:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e86007100 with x1876702014920064 ext(3145728->4194303) 00002000:00010000:2.0:1789762763.274508:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e86007100 00000001:00100000:0.0:1789762763.335406:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000001:00080000:3.0:1789762763.348163:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967363 is committed 00000100:00100000:2.0:1789762763.348213:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2365:x1876702014920448:12345-192.168.203.48@tcp:4:dd.0 Request processed in 26205us (109876us total) trans 4294967363 rc 0/0 00000100:00100000:2.0:1789762763.348223:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 235 00002000:00100000:2.0:1789762763.348233:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x8:0x0] (0->1048575) req@ffff981ebac6f100 x1876702014920448/t4294967363(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.348246:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac6f100 with x1876702014920448 ext(0->1048575) 00002000:00010000:2.0:1789762763.348253:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac6f100 00000100:00100000:3.0:1789762763.350559:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.350686:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014920704 00000100:00100000:2.0:1789762763.350694:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014920704 00002000:00100000:2.0:1789762763.350742:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0x8:0x0] (2097152->3145727) req@ffff981ebc554000 x1876702014920704/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.350758:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc554000 with x1876702014920704 ext(2097152->3145727) 00002000:00010000:2.0:1789762763.350764:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc554000 00000100:00100000:2.0:1789762763.350788:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286675876429 time 286625876429 state 0x1 ref 2 00000100:00100000:0.0:1789762763.357016:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.363463:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.363603:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014920960 00000100:00100000:2.0:1789762763.363615:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014920960 00002000:00100000:2.0:1789762763.363678:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0xc:0x0] (4194304->5242879) req@ffff981d84540380 x1876702014920960/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.363709:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d84540380 with x1876702014920960 ext(4194304->5242879) 00002000:00010000:2.0:1789762763.363714:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d84540380 00000100:00100000:2.0:1789762763.363730:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286675876429 time 286625876429 state 0x1 ref 4 00000100:00100000:3.0:1789762763.370064:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.370106:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014921088 00000100:00100000:2.0:1789762763.370112:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014921088 00002000:00100000:2.0:1789762763.370132:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0xc:0x0] (5242880->6291455) req@ffff981ebc555500 x1876702014921088/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.370140:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc555500 with x1876702014921088 ext(5242880->6291455) 00002000:00010000:2.0:1789762763.370143:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc555500 00000100:00100000:2.0:1789762763.370158:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286675876429 time 286625876429 state 0x1 ref 5 00000100:00100000:3.0:1789762763.377083:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.383755:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.390140:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.390204:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014921472 00000100:00100000:3.0:1789762763.390211:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014921472 00002000:00100000:3.0:1789762763.390238:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x8:0x0] (4194304->5242879) req@ffff981d84543800 x1876702014921472/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762763.390265:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d84543800 with x1876702014921472 ext(4194304->5242879) 00002000:00010000:3.0:1789762763.390269:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d84543800 00000100:00100000:3.0:1789762763.390283:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286725956960 time 286675956960 state 0x1 ref 7 00000100:00100000:3.0:1789762763.396966:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.428989:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.429132:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014921728 00000100:00100000:2.0:1789762763.429145:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014921728 00002000:00100000:2.0:1789762763.429191:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0x8:0x0] (6291456->7340031) req@ffff981ebc556680 x1876702014921728/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.429206:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc556680 with x1876702014921728 ext(6291456->7340031) 00002000:00010000:2.0:1789762763.429212:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc556680 00000100:00100000:2.0:1789762763.429249:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286776050882 time 286726050882 state 0x1 ref 8 00000100:00100000:0.0:1789762763.435556:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.435606:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.435672:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014922112 00000100:00100000:3.0:1789762763.435680:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014922112 00000100:00100000:3.0:1789762763.435740:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 55 00000100:00100000:3.0:1789762763.435768:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13027:x1876702014922112:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:3.0:1789762763.436059:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:13027:x1876702014922112:12345-192.168.203.48@tcp:35:dd.0 Request processed in 301us (450us total) trans 4294967453 rc 0/0 00000100:00100000:3.0:1789762763.436069:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 55 00000100:00100000:3.0:1789762763.465744:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:3.0:1789762763.468299:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b03139 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.468317:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b03139 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.468327:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b03139 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.468334:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762763.468339:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b03139 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.468366:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b03139 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.468381:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x992ad74f (ffff981ec2179700) 00010000:00010000:3.0:1789762763.468385:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762763.468389:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b03139 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.468397:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b03139 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.472231:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286826127455 time 286776127455 state 0x1 ref 9 00000100:00100000:2.0:1789762763.472256:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:2363:x1876702014920832:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762763.472708:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.472906:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014922496 00000100:00100000:0.0:1789762763.472918:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014922496 00000100:00100000:3.0:1789762763.472947:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.472994:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014922624 00000100:00100000:0.0:1789762763.473001:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014922624 00000100:00100000:0.0:1789762763.473075:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 128 00000100:00100000:0.0:1789762763.473157:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:13038:x1876702014922624:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762763.473176:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762763.473198:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000addc69eb/0x416193e012b0314e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd3e6 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762763.473250:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x9:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789762763.473273:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=286s v=(0 0) 00010000:00010000:0.0:1789762763.473281:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762763.473294:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000addc69eb/0x416193e012b0314e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3e6 expref: 19 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.473373:0:6647:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000addc69eb/0x416193e012b0314e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x70000000000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3e6 expref: 19 pid: 6647 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.473403:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000addc69eb/0x416193e012b0314e lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000000000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3e6 expref: 19 pid: 6647 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.473423:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86184600, rc 0) 00000100:00100000:0.0:1789762763.473491:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:13038:x1876702014922624:12345-192.168.203.48@tcp:101:dd.0 Request processed in 395us (541us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762763.473509:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 128 00000100:00100000:3.0:1789762763.475502:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.476585:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.476729:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014922880 00000100:00100000:2.0:1789762763.476744:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014922880 00000100:00100000:2.0:1789762763.476819:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 130 00000100:00100000:2.0:1789762763.476840:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:13039:x1876702014922880:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762763.476861:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762763.476883:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004b9f9a09/0x416193e012b03163 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd3f4 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.476916:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x9:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762763.476923:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762763.476934:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004b9f9a09/0x416193e012b03163 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3f4 expref: 20 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.476972:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004b9f9a09/0x416193e012b03163 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3f4 expref: 20 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.476986:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbe000, rc 0) 00000100:00100000:2.0:1789762763.477138:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:13039:x1876702014922880:12345-192.168.203.48@tcp:101:dd.0 Request processed in 254us (509us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762763.477152:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 130 00000100:00100000:3.0:1789762763.478867:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.478957:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014923008 00000100:00100000:2.0:1789762763.478968:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014923008 00000100:00100000:2.0:1789762763.479020:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 131 00000100:00100000:2.0:1789762763.479032:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:13047:x1876702014923008:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762763.479045:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762763.479059:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078bd4209/0x416193e012b0316a lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd3fb expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.479122:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x9:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762763.479130:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762763.479137:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078bd4209/0x416193e012b0316a lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3fb expref: 21 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.479153:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000078bd4209/0x416193e012b0316a lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3fb expref: 21 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.479167:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbf400, rc 0) 00000100:00100000:2.0:1789762763.479217:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:13047:x1876702014923008:12345-192.168.203.48@tcp:101:dd.0 Request processed in 186us (352us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762763.479238:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 131 00000100:00100000:0.0:1789762763.484680:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.484798:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014923136 00000100:00100000:2.0:1789762763.484809:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014923136 00000100:00100000:2.0:1789762763.484870:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 132 00000100:00100000:2.0:1789762763.484885:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:13049:x1876702014923136:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762763.484900:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1789762763.484913:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762763.484921:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b03171 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd402 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762763.484949:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xd:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789762763.484968:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=286s v=(0 0) 00010000:00010000:2.0:1789762763.484977:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762763.484991:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b03171 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd402 expref: 20 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.485013:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b03171 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4194304->5242879) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd402 expref: 20 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.485028:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbe800, rc 0) 00000100:00100000:2.0:1789762763.485138:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:13049:x1876702014923136:12345-192.168.203.48@tcp:101:dd.0 Request processed in 204us (410us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762763.485155:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 132 00000100:00100000:0.0:1789762763.487258:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.489277:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.489370:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.489411:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014923520 00000100:00100000:2.0:1789762763.489423:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014923520 00000100:00100000:2.0:1789762763.489467:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014923648 00000100:00100000:2.0:1789762763.489471:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014923648 00000100:00100000:2.0:1789762763.489526:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 134 00000100:00100000:2.0:1789762763.489540:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:13043:x1876702014923520:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762763.489560:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762763.489580:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0317f lrc: 2/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd410 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.489613:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0xd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789762763.489623:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762763.489670:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0317f lrc: 3/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd410 expref: 22 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.489748:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0317f lrc: 3/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd410 expref: 23 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.489789:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbf600, rc 0) 00000100:00100000:2.0:1789762763.489847:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:13043:x1876702014923520:12345-192.168.203.48@tcp:101:dd.0 Request processed in 303us (566us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762763.489885:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 134 00000100:00100000:2.0:1789762763.492609:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.495040:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.495733:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.495838:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014924032 00000100:00100000:3.0:1789762763.495848:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014924032 00000100:00100000:3.0:1789762763.495918:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 138 00000100:00100000:3.0:1789762763.495939:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86006300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:13044:x1876702014924032:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762763.495958:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762763.495981:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0319b lrc: 2/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd42c expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.496019:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0xd:0x0].0x0 since=0s v=(4 0) 00010000:00010000:3.0:1789762763.496029:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(5 0) 00010000:00010000:3.0:1789762763.496044:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0319b lrc: 3/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd42c expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.496127:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0319b lrc: 3/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd42c expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762763.496145:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e80031000, rc 0) 00000100:00100000:3.0:1789762763.496217:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86006300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:13044:x1876702014924032:12345-192.168.203.48@tcp:101:dd.0 Request processed in 281us (487us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762763.496236:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 138 00000100:00100000:0.0:1789762763.496755:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2366:x1876702014923392:12345-192.168.203.48@tcp:4:dd.0 Request processed in 9201us (9495us total) trans 4294967367 rc 0/0 00000100:00100000:0.0:1789762763.496770:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 2 00000100:00100000:0.0:1789762763.496783:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45a00 gen 0 rate 20 depth 3 token 2 deadline 286841401038 time 286791442568 state 0x2 ref 0 00002000:00100000:0.0:1789762763.496800:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x9:0x0] (1048576->2097151) req@ffff981ebfdf8a80 x1876702014923392/t4294967367(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762763.496814:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdf8a80 with x1876702014923392 ext(1048576->2097151) 00010000:00010000:0.0:1789762763.496825:0:12581:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000addc69eb/0x416193e012b0314e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3e6 expref: 24 pid: 6647 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.496862:0:12581:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000addc69eb/0x416193e012b0314e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 7 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3e6 expref: 24 pid: 6647 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762763.496872:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981ebfdf8a80 00000100:00100000:2.0:1789762763.497507:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.497810:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.497892:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014924288 00000100:00100000:0.0:1789762763.497904:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014924288 00000100:00100000:0.0:1789762763.497971:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 140 00000100:00100000:0.0:1789762763.497991:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:13042:x1876702014924288:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762763.498006:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762763.498022:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b031a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd43a expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.498047:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0xd:0x0].0x0 since=0s v=(5 0) 00010000:00010000:0.0:1789762763.498054:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762763.498062:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b031a9 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 8 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd43a expref: 25 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.498078:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b031a9 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 8 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd43a expref: 25 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.498131:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86185800, rc 0) 00000100:00100000:0.0:1789762763.498173:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:13042:x1876702014924288:12345-192.168.203.48@tcp:101:dd.0 Request processed in 190us (375us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762763.498184:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 140 00000100:00100000:2.0:1789762763.498511:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.498567:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014924416 00000100:00100000:3.0:1789762763.498576:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014924416 00000100:00100000:3.0:1789762763.498651:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 141 00000100:00100000:3.0:1789762763.498671:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:13051:x1876702014924416:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762763.498689:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762763.498711:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b031b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd441 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.498730:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762763.498942:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0xd:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762763.498968:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762763.498988:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b031b0 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd441 expref: 26 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.499019:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b031b0 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd441 expref: 26 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.499055:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e80031a00, rc 0) 00000100:00100000:0.0:1789762763.499164:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:13051:x1876702014924416:12345-192.168.203.48@tcp:101:dd.0 Request processed in 494us (653us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762763.499182:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 141 00000100:00100000:2.0:1789762763.499207:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.499227:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014924672 00000100:00100000:0.0:1789762763.499239:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014924672 00000100:00100000:0.0:1789762763.499304:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 142 00000100:00100000:0.0:1789762763.499332:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:13046:x1876702014924672:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762763.499353:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762763.499372:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006f212c1e/0x416193e012b031b7 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 10 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd448 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.499518:0:6648:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000ce52c272/0x416193e012b031a2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd433 expref: 26 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.499532:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x9:0x0].0x0 since=0s v=(6 0) 00010000:00010000:0.0:1789762763.499536:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762763.499542:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006f212c1e/0x416193e012b031b7 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd448 expref: 26 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.499609:0:6648:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000ce52c272/0x416193e012b031a2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 10 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x50000000020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd433 expref: 26 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.499665:0:6648:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000ce52c272/0x416193e012b031a2 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 10 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x70000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd433 expref: 26 pid: 10916 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762763.499696:0:6648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6648:1876702022154752:192.168.203.48@tcp:104: 00000100:00100000:0.0:1789762763.499778:0:6648:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebc2efa80 going to sleep for 11 seconds 00000100:00100000:0.0:1789762763.501162:0:6648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfa300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6648:1876702022154752:192.168.203.48@tcp:104: 00010000:00010000:0.0:1789762763.501197:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006f212c1e/0x416193e012b031b7 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd448 expref: 26 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.501219:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86184800, rc 0) 00000100:00100000:0.0:1789762763.501284:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:13046:x1876702014924672:12345-192.168.203.48@tcp:101:dd.0 Request processed in 1965us (2087us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762763.501295:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 142 00000100:00100000:3.0:1789762763.522446:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286876335426 time 286826335426 state 0x1 ref 8 00000100:00100000:3.0:1789762763.522482:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2363:x1876702014920960:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762763.525319:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.525660:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.525775:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014924928 00000100:00100000:0.0:1789762763.525788:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014924928 00002000:00100000:0.0:1789762763.525829:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x9:0x0] (7340032->8388607) req@ffff981ebc4f5f80 x1876702014924928/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762763.525844:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc4f5f80 with x1876702014924928 ext(7340032->8388607) 00002000:00010000:0.0:1789762763.525850:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc4f5f80 00000100:00100000:0.0:1789762763.525880:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286876335426 time 286826335426 state 0x1 ref 9 00000100:00100000:0.0:1789762763.530986:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.531062:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014925056 00000100:00100000:3.0:1789762763.531073:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014925056 00010000:00010000:3.0:1789762763.531143:0:5790:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000ce52c272/0x416193e012b031a2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd433 expref: 28 pid: 10916 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762763.531246:0:8405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 56 00000100:00100000:0.0:1789762763.531270:0:8405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86f02050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2366:x1876702014925056:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762763.531288:0:8405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1789762763.531308:0:8405:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x9:0x0] updating lvb size from disk: 2097152 -> 5242880 00010000:00010000:0.0:1789762763.531322:0:8405:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000ce52c272/0x416193e012b031a2 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd433 expref: 28 pid: 10916 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.531346:0:8405:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000ce52c272/0x416193e012b031a2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x50000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd433 expref: 28 pid: 10916 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.531378:0:8405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ce52c272/0x416193e012b031a2 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 9 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x44801400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd433 expref: 28 pid: 10916 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.531402:0:8405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762763.531409:0:8405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x9:0x0].0x0 since=0s v=(7 0) 00010000:00010000:0.0:1789762763.531422:0:8405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006f212c1e/0x416193e012b031b7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd448 expref: 27 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.531439:0:8405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006f212c1e/0x416193e012b031b7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd448 expref: 27 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.531487:0:8405:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000006f212c1e/0x416193e012b031b7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd448 expref: 27 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762763.531532:0:8405:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:8405:1876702022155520:192.168.203.48@tcp:105: 00000100:00100000:0.0:1789762763.531666:0:8405:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebc2efa80 going to sleep for 11 seconds 00000100:00100000:0.0:1789762763.532712:0:8405:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfb480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_005:lustre-OST0001_UUID:8405:1876702022155520:192.168.203.48@tcp:105: 00010000:00010000:0.0:1789762763.532737:0:8405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762763.532803:0:8405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86f02050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2366:x1876702014925056:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 1539us (1819us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762763.532820:0:8405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 56 00000100:00100000:2.0:1789762763.537551:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.537688:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014925184 00000100:00100000:0.0:1789762763.537700:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014925184 00002000:00100000:0.0:1789762763.537744:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x9:0x0] (0->1048575) req@ffff981ebfc03b80 x1876702014925184/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762763.537764:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfc03b80 with x1876702014925184 ext(0->1048575) 00002000:00010000:0.0:1789762763.537772:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfc03b80 00000100:00100000:0.0:1789762763.537798:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286876335426 time 286826335426 state 0x1 ref 9 00000100:00100000:3.0:1789762763.539068:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.539158:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014925312 00000100:00100000:0.0:1789762763.539168:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014925312 00002000:00100000:0.0:1789762763.539212:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xd:0x0] (4194304->5242879) req@ffff981ebc4f7b80 x1876702014925312/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:3.0:1789762763.539241:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00002000:00100000:0.0:1789762763.539243:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc4f7b80 with x1876702014925312 ext(4194304->5242879) 00010000:00010000:0.0:1789762763.539263:0:6653:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b03171 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 4194304->5242879) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd402 expref: 27 pid: 6647 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762763.539314:0:6653:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b03171 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 4194304->5242879) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd402 expref: 27 pid: 6647 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762763.539333:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff981ebc4f7b80 00000100:00100000:0.0:1789762763.539355:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45a00 gen 0 rate 20 depth 3 token 1 deadline 286841442568 time 286829635160 state 0x0 ref 1 00000100:00100000:2.0:1789762763.539376:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014925440 00000100:00100000:0.0:1789762763.539386:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45a00 gen 0 rate 20 depth 3 token 1 deadline 286879635160 time 286829635160 state 0x1 ref 1 00000100:00100000:2.0:1789762763.539388:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014925440 00002000:00100000:2.0:1789762763.539427:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0x9:0x0] (3145728->4194303) req@ffff981ebc4f6a00 x1876702014925440/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.539445:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc4f6a00 with x1876702014925440 ext(3145728->4194303) 00002000:00010000:2.0:1789762763.539457:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc4f6a00 00000100:00100000:0.0:1789762763.539460:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45a00 gen 0 rate 20 depth 3 token 0 deadline 286879635160 time 286843350654 state 0x0 ref 1 00000100:00100000:0.0:1789762763.539494:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2363:x1876702014925312:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762763.539497:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286876335426 time 286826335426 state 0x1 ref 10 00000100:00100000:0.0:1789762763.543182:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014925568 00000100:00100000:0.0:1789762763.543194:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014925568 00002000:00100000:0.0:1789762763.543240:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x9:0x0] (2097152->3145727) req@ffff981ebad01880 x1876702014925568/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762763.543258:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad01880 with x1876702014925568 ext(2097152->3145727) 00002000:00010000:0.0:1789762763.543267:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebad01880 00000100:00100000:0.0:1789762763.543291:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286876335426 time 286826335426 state 0x1 ref 11 00000100:00100000:2.0:1789762763.548693:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014925696 00000100:00100000:2.0:1789762763.548705:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014925696 00010000:00010000:2.0:1789762763.548745:0:5791:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b03171 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 4194304->5242879) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd402 expref: 29 pid: 6647 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.548798:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 57 00000100:00100000:2.0:1789762763.548809:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2366:x1876702014925696:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762763.548823:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1789762763.548836:0:5791:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xd:0x0] updating lvb size from disk: 0 -> 5242880 00002000:00010000:2.0:1789762763.548840:0:5791:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0xd:0x0] updating lvb mtime from disk: 0 -> 1789762763 00002000:00010000:2.0:1789762763.548844:0:5791:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0xd:0x0] updating lvb ctime from disk: 0 -> 1789762763 00010000:00010000:2.0:1789762763.548852:0:5791:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b03171 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 4194304->5242879) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd402 expref: 29 pid: 6647 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.548869:0:5791:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b03171 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 4194304->5242879) gid 0 flags: 0x50000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd402 expref: 29 pid: 6647 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.548892:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b03171 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 4194304->5242879) gid 0 flags: 0x44801400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd402 expref: 29 pid: 6647 timeout: 386 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.548919:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.548924:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.548933:0:5791:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03178 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd409 expref: 28 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.548946:0:5791:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03178 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd409 expref: 28 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.548956:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.548959:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.548965:0:5791:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0317f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd410 expref: 28 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.548975:0:5791:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0317f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd410 expref: 28 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.548983:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.548986:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.548991:0:5791:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b03186 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd417 expref: 28 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.549001:0:5791:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b03186 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd417 expref: 28 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.549008:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.549017:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.549024:0:5791:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b03194 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd425 expref: 28 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.549034:0:5791:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b03194 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd425 expref: 28 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.549042:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.549045:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.549051:0:5791:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0319b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [3145728->5242879] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd42c expref: 28 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.549060:0:5791:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0319b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [3145728->5242879] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd42c expref: 28 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.549067:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.549070:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.549075:0:5791:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b031a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd43a expref: 28 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.549125:0:5791:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b031a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd43a expref: 28 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.549135:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.549138:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762763.549145:0:5791:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b031b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd441 expref: 28 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.549172:0:5791:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b031b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd441 expref: 28 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762763.549207:0:5791:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b031b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd441 expref: 28 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.549221:0:5791:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc02300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022155648:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762763.549305:0:5791:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b031a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd43a expref: 28 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.549319:0:5791:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc01880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022155776:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762763.549358:0:5791:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0319b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [3145728->5242879] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd42c expref: 28 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.549371:0:5791:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc00380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022155904:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762763.549410:0:5791:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b03194 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd425 expref: 28 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.549420:0:5791:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc03100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022156032:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762763.549454:0:5791:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b03186 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd417 expref: 28 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.549487:0:5791:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfc00e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022156160:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762763.549548:0:5791:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0317f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd410 expref: 28 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.549563:0:5791:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e8616c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022156288:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762763.549598:0:5791:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03178 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd409 expref: 28 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762763.549612:0:5791:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e8616fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022156416:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762763.549658:0:5791:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebfc28780 going to sleep for 11 seconds 00000100:00100000:2.0:1789762763.550586:0:5791:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc02300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022155648:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762763.550602:0:5791:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc01880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022155776:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762763.551584:0:5791:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc00380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022155904:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762763.551597:0:5791:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc03100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022156032:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762763.551605:0:5791:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfc00e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022156160:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762763.551614:0:5791:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e8616c380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022156288:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762763.567106:0:5791:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e8616fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5791:1876702022156416:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762763.567159:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762763.567208:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2366:x1876702014925696:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 18401us (18674us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762763.567234:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 57 00000100:00100000:0.0:1789762763.573006:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.573066:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014925824 00000100:00100000:2.0:1789762763.573075:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014925824 00002000:00100000:2.0:1789762763.573133:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0xd:0x0] (1048576->2097151) req@ffff981ebfdfb480 x1876702014925824/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.573145:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfb480 with x1876702014925824 ext(1048576->2097151) 00002000:00010000:2.0:1789762763.573150:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdfb480 00000100:00100000:2.0:1789762763.573166:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 286926480327 time 286876480327 state 0x1 ref 12 00000100:00100000:2.0:1789762763.576193:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.586935:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.587894:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.589751:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.591760:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.592907:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.672875:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287026766388 time 286976766388 state 0x1 ref 16 00000100:00100000:2.0:1789762763.672908:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2363:x1876702014921344:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762763.676780:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2363:x1876702014921344:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3881us (293023us total) trans 4294967682 rc 0/0 00000100:00100000:2.0:1789762763.676811:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 242 00002000:00100000:2.0:1789762763.676827:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0xc:0x0] (7340032->8388607) req@ffff981d84541180 x1876702014921344/t4294967682(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.676842:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d84541180 with x1876702014921344 ext(7340032->8388607) 00002000:00010000:2.0:1789762763.676850:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d84541180 00000100:00100000:2.0:1789762763.728861:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762763.728936:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014926848 00000100:00100000:0.0:1789762763.728946:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014926848 00000100:00100000:0.0:1789762763.728994:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 56 00000100:00100000:0.0:1789762763.729006:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:13047:x1876702014926848:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:0.0:1789762763.729279:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:13047:x1876702014926848:12345-192.168.203.48@tcp:35:dd.0 Request processed in 273us (417us total) trans 4294967455 rc 0/0 00000100:00100000:0.0:1789762763.729290:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 56 00000100:00100000:0.0:1789762763.729595:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762763.729663:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014926976 00000100:00100000:2.0:1789762763.729668:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014926976 00002000:00100000:2.0:1789762763.729693:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0x9:0x0] (6291456->7340031) req@ffff981ebfdfaa00 x1876702014926976/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.729703:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfaa00 with x1876702014926976 ext(6291456->7340031) 00002000:00010000:2.0:1789762763.729707:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdfaa00 00000100:00100000:2.0:1789762763.729720:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287076873175 time 287026873175 state 0x1 ref 15 00000100:00100000:2.0:1789762763.730591:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762763.730681:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014927104 00000100:00100000:3.0:1789762763.730706:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014927104 00002000:00100000:3.0:1789762763.730732:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x9:0x0] (5242880->6291455) req@ffff981e8616ce00 x1876702014927104/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/0 e 0 to 0 dl 1789762774 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762763.730794:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e8616ce00 with x1876702014927104 ext(5242880->6291455) 00002000:00010000:3.0:1789762763.730798:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e8616ce00 00000100:00100000:3.0:1789762763.730831:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287076873175 time 287026873175 state 0x1 ref 16 00000100:00100000:2.0:1789762763.773181:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287127069927 time 287077069927 state 0x1 ref 16 00000100:00100000:2.0:1789762763.773216:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc557480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:2364:x1876702014921600:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762763.784833:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc557480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2364:x1876702014921600:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11623us (387861us total) trans 4294967370 rc 0/0 00000100:00100000:3.0:1789762763.784849:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 244 00002000:00100000:3.0:1789762763.784867:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x8:0x0] (5242880->6291455) req@ffff981ebc557480 x1876702014921600/t4294967370(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762763.784883:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc557480 with x1876702014921600 ext(5242880->6291455) 00002000:00010000:3.0:1789762763.784892:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc557480 00000100:00100000:2.0:1789762763.873488:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287227377459 time 287177377459 state 0x1 ref 14 00000100:00100000:2.0:1789762763.873521:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2364:x1876702014921856:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762763.877424:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2364:x1876702014921856:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3910us (441867us total) trans 4294967372 rc 0/0 00000100:00100000:2.0:1789762763.877437:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 246 00002000:00100000:2.0:1789762763.877451:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0x8:0x0] (7340032->8388607) req@ffff981d84543100 x1876702014921856/t4294967372(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762763.877464:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d84543100 with x1876702014921856 ext(7340032->8388607) 00002000:00010000:2.0:1789762763.877473:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d84543100 00000100:00100000:3.0:1789762763.973750:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287327647710 time 287277647710 state 0x1 ref 12 00000100:00100000:3.0:1789762763.973773:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2364:x1876702014925184:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762763.977017:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2364:x1876702014925184:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3249us (439468us total) trans 4294967374 rc 0/0 00000100:00100000:3.0:1789762763.977038:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 248 00002000:00100000:3.0:1789762763.977048:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x9:0x0] (0->1048575) req@ffff981ebfc03b80 x1876702014925184/t4294967374(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762763.977058:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfc03b80 with x1876702014925184 ext(0->1048575) 00002000:00010000:3.0:1789762763.977064:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfc03b80 00000100:00100000:3.0:1789762764.028181:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2365:x1876702014925440:12345-192.168.203.48@tcp:4:dd.0 Request processed in 4256us (488946us total) trans 4294967375 rc 0/0 00000100:00100000:3.0:1789762764.028202:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 249 00002000:00100000:3.0:1789762764.028223:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x9:0x0] (3145728->4194303) req@ffff981ebc4f6a00 x1876702014925440/t4294967375(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762764.028244:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc4f6a00 with x1876702014925440 ext(3145728->4194303) 00002000:00010000:3.0:1789762764.028252:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc4f6a00 00000100:00100000:3.0:1789762764.073942:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287427840222 time 287377840222 state 0x1 ref 10 00000100:00100000:3.0:1789762764.073964:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2364:x1876702014925568:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762764.077020:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2364:x1876702014925568:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3060us (533943us total) trans 4294967376 rc 0/0 00000100:00100000:3.0:1789762764.077030:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 250 00002000:00100000:3.0:1789762764.077043:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x9:0x0] (2097152->3145727) req@ffff981ebad01880 x1876702014925568/t4294967376(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762764.077056:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad01880 with x1876702014925568 ext(2097152->3145727) 00002000:00010000:3.0:1789762764.077063:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebad01880 00000100:00100000:0.0:1789762764.127687:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2363:x1876702014925824:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3511us (554684us total) trans 4294967683 rc 0/0 00000100:00100000:0.0:1789762764.127695:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 251 00002000:00100000:0.0:1789762764.127704:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xd:0x0] (1048576->2097151) req@ffff981ebfdfb480 x1876702014925824/t4294967683(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762764.127712:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfb480 with x1876702014925824 ext(1048576->2097151) 00002000:00010000:0.0:1789762764.127717:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdfb480 00000100:00100000:2.0:1789762764.174361:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287528251761 time 287478251761 state 0x1 ref 8 00000100:00100000:2.0:1789762764.174407:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702014925952:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762764.185821:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2365:x1876702014925952:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11417us (609627us total) trans 4294967684 rc 0/0 00000100:00100000:3.0:1789762764.185837:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 252 00002000:00100000:3.0:1789762764.185854:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0xd:0x0] (2097152->3145727) req@ffff981e8616fb80 x1876702014925952/t4294967684(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762764.185870:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e8616fb80 with x1876702014925952 ext(2097152->3145727) 00002000:00010000:3.0:1789762764.185878:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e8616fb80 00000100:00100000:0.0:1789762764.228388:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2366:x1876702014926080:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3869us (641451us total) trans 4294967685 rc 0/0 00000100:00100000:0.0:1789762764.228403:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 253 00002000:00100000:0.0:1789762764.228419:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xd:0x0] (7340032->8388607) req@ffff981ebfdf8700 x1876702014926080/t4294967685(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762764.228435:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdf8700 with x1876702014926080 ext(7340032->8388607) 00002000:00010000:0.0:1789762764.228442:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdf8700 00000100:00100000:3.0:1789762764.274615:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287628511709 time 287578511709 state 0x1 ref 6 00000100:00100000:3.0:1789762764.274658:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2365:x1876702014926208:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762764.278615:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2365:x1876702014926208:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3978us (690719us total) trans 4294967686 rc 0/0 00000100:00100000:3.0:1789762764.278657:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 254 00002000:00100000:3.0:1789762764.278690:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0xd:0x0] (0->1048575) req@ffff981e8616c380 x1876702014926208/t4294967686(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762764.278702:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e8616c380 with x1876702014926208 ext(0->1048575) 00002000:00010000:3.0:1789762764.278708:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e8616c380 00000100:00100000:3.0:1789762764.324753:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287678644980 time 287628644980 state 0x1 ref 5 00000100:00100000:3.0:1789762764.324785:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2364:x1876702014926336:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762764.329824:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:2364:x1876702014926336:12345-192.168.203.48@tcp:4:dd.0 Request processed in 5046us (740070us total) trans 4294967687 rc 0/0 00000100:00100000:3.0:1789762764.329837:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 255 00002000:00100000:3.0:1789762764.329862:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xd:0x0] (3145728->4194303) req@ffff981ebfdf9c00 x1876702014926336/t4294967687(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762764.329876:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdf9c00 with x1876702014926336 ext(3145728->4194303) 00002000:00010000:3.0:1789762764.329883:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdf9c00 00000100:00100000:3.0:1789762764.374845:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287728736568 time 287678736568 state 0x1 ref 4 00000100:00100000:3.0:1789762764.374874:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2365:x1876702014926464:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762764.380278:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:2365:x1876702014926464:12345-192.168.203.48@tcp:4:dd.0 Request processed in 5409us (788515us total) trans 4294967688 rc 0/0 00000100:00100000:3.0:1789762764.380291:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 256 00002000:00100000:3.0:1789762764.380306:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0xd:0x0] (6291456->7340031) req@ffff981ebfdfbb80 x1876702014926464/t4294967688(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762764.380321:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfbb80 with x1876702014926464 ext(6291456->7340031) 00002000:00010000:3.0:1789762764.380330:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdfbb80 00000100:00100000:0.0:1789762764.425013:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287778899222 time 287728899222 state 0x1 ref 3 00000100:00100000:0.0:1789762764.425050:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:2363:x1876702014926592:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762764.435262:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+45:2363:x1876702014926592:12345-192.168.203.48@tcp:4:dd.0 Request processed in 10220us (842356us total) trans 4294967689 rc 0/0 00000100:00100000:0.0:1789762764.435274:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 257 00002000:00100000:0.0:1789762764.435286:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xd:0x0] (5242880->6291455) req@ffff981e8616f800 x1876702014926592/t4294967689(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762764.435298:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e8616f800 with x1876702014926592 ext(5242880->6291455) 00002000:00010000:0.0:1789762764.435305:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e8616f800 00000100:00100000:2.0:1789762764.439512:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762764.475195:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287829086285 time 287779086285 state 0x1 ref 2 00000100:00100000:3.0:1789762764.475224:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2364:x1876702014926976:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762764.479146:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2364:x1876702014926976:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3925us (749543us total) trans 4294967377 rc 0/0 00000100:00100000:3.0:1789762764.479164:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 258 00002000:00100000:3.0:1789762764.479184:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0x9:0x0] (6291456->7340031) req@ffff981ebfdfaa00 x1876702014926976/t4294967377(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:544/0 lens 488/448 e 0 to 0 dl 1789762774 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762764.479223:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfaa00 with x1876702014926976 ext(6291456->7340031) 00002000:00010000:3.0:1789762764.479231:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdfaa00 00000100:00100000:0.0:1789762764.525305:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287829086285 time 287829198235 state 0x0 ref 1 00000100:00100000:0.0:1789762764.525335:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2364:x1876702014927104:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762764.532511:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762764.532581:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014927360 00000100:00100000:0.0:1789762764.532593:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014927360 00000100:00100000:0.0:1789762764.532664:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 13 00000100:00100000:0.0:1789762764.532676:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:5592:x1876702014927360:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762764.532690:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 7 locks, starting at 0 00002000:00010000:0.0:1789762764.532704:0:5790:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x8:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:0.0:1789762764.532707:0:5790:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x8:0x0] updating lvb mtime from disk: 0 -> 1789762763 00002000:00010000:0.0:1789762764.532710:0:5790:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x8:0x0] updating lvb ctime from disk: 0 -> 1789762763 00002000:00010000:0.0:1789762764.532712:0:5790:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x8:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:0.0:1789762764.532733:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cbe3ac7c/0x416193e012b03124 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3d1 expref: 42 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762764.532771:0:5790:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x9:0x0] updating lvb size from disk: 5242880 -> 8388608 00010000:00010000:0.0:1789762764.532779:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006f212c1e/0x416193e012b031b7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd448 expref: 41 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.532803:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078bd4209/0x416193e012b0316a lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 7 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3fb expref: 40 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.532814:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004b9f9a09/0x416193e012b03163 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3f4 expref: 39 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.532823:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e8859828/0x416193e012b0315c lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3ed expref: 38 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.532833:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b03155 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 4 type: EXT [0->2097151] (req 0->1048575) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3df expref: 37 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.532843:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000032976790/0x416193e012b0318d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 3 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd41e expref: 36 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.532853:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762764.532888:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:5592:x1876702014927360:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 216us (380us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762764.532895:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 13 00000100:00100000:2.0:1789762764.570468:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762764.570535:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014927616 00000100:00100000:0.0:1789762764.570546:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014927616 00000100:00100000:0.0:1789762764.570597:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 199 00000100:00100000:0.0:1789762764.570607:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:13072:x1876702014927616:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762764.570647:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762764.570677:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184800/0x416193e012b031be lrc: 2/0,0 mode: --/CR res: [0x200000401:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd44f expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.570764:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b031c5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.570777:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b031c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.570786:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b031c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762764.570795:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x20:0x0]: returning size 16777216 00010000:00010000:0.0:1789762764.570895:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184800/0x416193e012b031be lrc: 0/0,0 mode: --/CR res: [0x200000401:0x1e:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd44f expref: 21 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.570911:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b031c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd44f expref: 20 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.570918:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff981e82a26500) 00010000:00010000:0.0:1789762764.570921:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762764.570924:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86185c00, rc 0) 00000100:00100000:0.0:1789762764.570971:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:13072:x1876702014927616:12345-192.168.203.48@tcp:101:dd.0 Request processed in 366us (511us total) trans 4294967456 rc 0/0 00000100:00100000:0.0:1789762764.570977:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 199 00000100:00100000:2.0:1789762764.571646:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762764.572671:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762764.572723:0:5792:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014928000 00000100:00100000:0.0:1789762764.572730:0:5792:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014928000 00000100:00100000:0.0:1789762764.572772:0:5792:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 14 00000100:00100000:0.0:1789762764.572779:0:5792:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:2364:x1876702014928000:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762764.572799:0:5792:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762764.572815:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b031c5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd44f expref: 22 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.572828:0:5792:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff981e82a26500) 00010000:00010000:0.0:1789762764.572831:0:5792:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762764.572862:0:5792:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:2364:x1876702014928000:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 83us (193us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762764.572869:0:5792:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 14 00000100:00100000:2.0:1789762764.573329:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762764.573448:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762764.573541:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014928384 00000100:00100000:3.0:1789762764.573605:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014928384 00000100:00100000:3.0:1789762764.573677:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 57 00000100:00100000:3.0:1789762764.573689:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:13074:x1876702014928384:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:2.0:1789762764.573697:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762764.573735:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014928512 00000100:00100000:0.0:1789762764.573743:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014928512 00000100:00100000:0.0:1789762764.573795:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 143 00000100:00100000:0.0:1789762764.573808:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:13072:x1876702014928512:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762764.573819:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762764.573835:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b031e8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd45d expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762764.573858:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xc:0x0] initial LVB size: 8388608, mtime: 0x6aad9ccb, atime: 0x0, ctime: 0x6aad9ccb, blocks: 0x1 00000100:00100000:3.0:1789762764.573868:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:13074:x1876702014928384:12345-192.168.203.48@tcp:35:dd.0 Request processed in 181us (425us total) trans 4294967458 rc 0/0 00010000:00010000:0.0:1789762764.573874:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=287s v=(0 0) 00000100:00100000:3.0:1789762764.573878:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 57 00010000:00010000:0.0:1789762764.573882:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762764.573893:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b031e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd45d expref: 38 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.573907:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b031e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd45d expref: 38 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.573930:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86184a00, rc 0) 00000100:00100000:0.0:1789762764.573966:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:13072:x1876702014928512:12345-192.168.203.48@tcp:101:dd.0 Request processed in 162us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762764.573974:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 143 00000100:00100000:2.0:1789762764.579604:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762764.579666:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014928640 00000100:00100000:3.0:1789762764.579673:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014928640 00002000:00100000:3.0:1789762764.579701:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0xc:0x0] (0->1048575) req@ffff981e8616d880 x1876702014928640/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:545/0 lens 488/0 e 0 to 0 dl 1789762775 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762764.579710:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e8616d880 with x1876702014928640 ext(0->1048575) 00002000:00010000:3.0:1789762764.579714:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e8616d880 00000100:00100000:3.0:1789762764.579729:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253333773379 time 253333704950 state 0x0 ref 1 00000100:00100000:3.0:1789762764.579736:0:12581:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 5: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 253333773379 time 253333704950 state 0x0 ref 1 00000100:00100000:3.0:1789762764.579750:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 3 deadline 287983657295 time 287883657295 state 0x1 ref 1 00000100:00100000:3.0:1789762764.579776:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 2 deadline 287983657295 time 287883684405 state 0x0 ref 1 00000100:00100000:3.0:1789762764.579799:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2364:x1876702014928640:12345-192.168.203.48@tcp:3:dd.0 00000100:00100000:3.0:1789762764.584435:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2364:x1876702014928640:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4636us (4832us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762764.584445:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 260 00000100:00100000:3.0:1789762764.584453:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 2 deadline 287983657295 time 287883684405 state 0x2 ref 0 00002000:00100000:3.0:1789762764.584463:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0xc:0x0] (0->1048575) req@ffff981e8616d880 x1876702014928640/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:545/0 lens 488/440 e 0 to 0 dl 1789762775 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762764.584472:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e8616d880 with x1876702014928640 ext(0->1048575) 00002000:00010000:3.0:1789762764.584476:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e8616d880 00000100:00100000:0.0:1789762764.620548:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762764.631328:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762764.632694:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762764.632742:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014929280 00000100:00100000:0.0:1789762764.632748:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014929280 00000100:00100000:0.0:1789762764.632780:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 202 00000100:00100000:0.0:1789762764.632786:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:13088:x1876702014929280:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:0.0:1789762764.632793:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762764.632802:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184800/0x416193e012b031fd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd46b expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762764.632842:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1e:0x0]: returning size 16384 00010000:00010000:0.0:1789762764.632894:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184800/0x416193e012b031fd lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd46b expref: 25 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.632912:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179000) 00010000:00010000:0.0:1789762764.632914:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762764.632917:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184800/0x416193e012b031fd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd46b expref: 25 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.632925:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86184800, rc 0) 00000100:00100000:0.0:1789762764.632971:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:13088:x1876702014929280:12345-192.168.203.48@tcp:101:rm.0 Request processed in 169us (262us total) trans 4294967460 rc 301/301 00000100:00100000:0.0:1789762764.632992:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 202 00000100:00100000:0.0:1789762764.634428:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762764.634468:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014929408 00000100:00100000:3.0:1789762764.634474:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014929408 00000100:00100000:3.0:1789762764.634502:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 59 00000100:00100000:3.0:1789762764.634509:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:13088:x1876702014929408:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:3.0:1789762764.636349:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:13088:x1876702014929408:12345-192.168.203.48@tcp:37:rm.0 Request processed in 1838us (1919us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762764.636358:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 59 00000100:00100000:2.0:1789762764.637590:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762764.638522:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762764.639565:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762764.640495:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762764.640524:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014930304 00000100:00100000:0.0:1789762764.640530:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014930304 00000100:00100000:0.0:1789762764.640558:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 205 00000100:00100000:0.0:1789762764.640565:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:13088:x1876702014930304:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:0.0:1789762764.640586:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1789762764.640599:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbee00/0x416193e012b03108 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd3c3 expref: 27 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.640612:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff981e82a26500) 00010000:00010000:0.0:1789762764.640638:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b031f6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd464 expref: 26 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.640645:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x0 (ffff981ea0aa3300) 00010000:00010000:0.0:1789762764.640647:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762764.640657:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b03220 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.640663:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b03220 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.640669:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b03220 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.640678:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b03227 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.640692:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b03227 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.640697:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b03227 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.640747:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b0322e lrc: 3/0,1 mode: --/EX res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.640752:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b0322e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x20:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.640756:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b0322e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x20:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762764.641271:0:5804:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].12, 64 off8960 00000004:00080000:0.0:1789762764.641279:0:5804:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 287945191197ns, next at 288945189829ns, rc = 0 00000040:00080000:0.0:1789762764.641403:0:5804:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].8, 64 off8704 00000004:00080000:0.0:1789762764.641409:0:5804:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 287945321509ns, next at 288945320691ns, rc = 0 00010000:00010000:0.0:1789762764.641445:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184600/0x416193e012b03235 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641452:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184600/0x416193e012b03235 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x20:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641458:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184600/0x416193e012b03235 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x20:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641463:0:5804:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184600/0x416193e012b03235 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x20:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641476:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184600/0x416193e012b03235 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x20:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641480:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184600/0x416193e012b03235 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x20:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641495:0:5804:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762764.641505:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b0322e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641509:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b0322e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641514:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b0322e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641519:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762764.641523:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b0322e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.641525:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762764.641529:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b0322e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.641532:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184600/0x416193e012b03235 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x20:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641538:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762764.641542:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762764.641545:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b0322e lrc: 2/0,0 mode: --/EX res: [0x200000401:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.641548:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762764.641550:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b0322e lrc: 0/0,0 mode: --/EX res: [0x200000401:0x20:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.641553:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184600/0x416193e012b03235 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x20:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641566:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b03220 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.641574:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184600/0x416193e012b03235 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x20:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641583:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b03220 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641589:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b03220 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.641590:0:5794:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762764.641596:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762764.641597:0:5794:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762764.641603:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b03220 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.641603:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184600/0x416193e012b03235 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x20:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.641621:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184600/0x416193e012b03235 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x20:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641622:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b03220 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.641656:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x0 (ffff981ea0aa3300) 00010000:00010000:0.0:1789762764.641658:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762764.641661:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b03220 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.641663:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762764.641666:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b03220 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.641667:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762764.641670:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00010000:0.0:1789762764.641681:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b330d46c reply state 00000000115d86da transno 4294967463 ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b03227 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762764.641687:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762764.641721:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc03100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:13088:x1876702014930304:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1157us (1225us total) trans 4294967463 rc 0/0 00000100:00100000:0.0:1789762764.641727:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 205 00010000:00010000:3.0:1789762764.642315:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b03227 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.642322:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b03227 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.642327:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b03227 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.642330:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762764.642333:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b03227 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.642337:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b03227 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.642344:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x77289514 (ffff981ea0aa3c00) 00010000:00010000:3.0:1789762764.642351:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762764.642354:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b03227 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.642358:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b03227 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762764.643735:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762764.644420:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762764.644427:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4400/0x416193e012b03251 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.644452:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762764.644458:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4400/0x416193e012b03251 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.644475:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4400/0x416193e012b03251 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.644496:0:6663:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:0.0:1789762764.644503:0:6663:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762764.644508:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4400/0x416193e012b03251 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.644515:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4400/0x416193e012b03251 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.644532:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762764.644535:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762764.644537:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762764.645029:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03243 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.645037:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03243 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.645042:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03243 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.645047:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762764.645049:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03243 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.645053:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03243 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.645060:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x992ad74f (ffff981ec2179400) 00010000:00010000:3.0:1789762764.645062:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762764.645064:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03243 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.645068:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03243 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x1e:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762764.645558:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762764.645588:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014930816 00000100:00100000:3.0:1789762764.645641:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014930816 00000100:00100000:3.0:1789762764.645668:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 61 00000100:00100000:3.0:1789762764.645675:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:13088:x1876702014930816:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762764.645762:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:13088:x1876702014930816:12345-192.168.203.48@tcp:35:rm.0 Request processed in 87us (206us total) trans 4294967465 rc 0/0 00000100:00100000:3.0:1789762764.645770:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 61 00000100:00100000:0.0:1789762764.646461:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:3.0:1789762764.647514:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b0325f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.647523:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b0325f lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.647528:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b0325f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.647531:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762764.647534:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b0325f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.647538:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b0325f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.647545:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179800) 00010000:00010000:3.0:1789762764.647547:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762764.647549:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b0325f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.647553:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b0325f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.727279:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762764.982320:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762764.982389:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014931200 00000100:00100000:3.0:1789762764.982408:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014931200 00000100:00100000:3.0:1789762764.982471:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 208 00000100:00100000:3.0:1789762764.982488:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:13099:x1876702014931200:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:3.0:1789762764.982516:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762764.982543:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b0326d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd479 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.982640:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b03274 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.982655:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b03274 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.982669:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b03274 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762764.982703:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762764.982707:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762764.982726:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b03274 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.982751:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b0326d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd479 expref: 27 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.982767:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b03274 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd479 expref: 26 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.982775:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46000) 00010000:00010000:3.0:1789762764.982779:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762764.982783:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbe200, rc 0) 00000100:00100000:3.0:1789762764.982858:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:13099:x1876702014931200:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 371us (543us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762764.982869:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 208 00000100:00100000:2.0:1789762764.984395:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762764.984439:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014931456 00000100:00100000:3.0:1789762764.984446:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014931456 00000100:00100000:3.0:1789762764.984487:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 209 00000100:00100000:3.0:1789762764.984496:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc02300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:13099:x1876702014931456:12345-192.168.203.48@tcp:36:mkdir.0 00010000:00010000:3.0:1789762764.984530:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762764.984546:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b03274 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd479 expref: 26 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.984562:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46000) 00010000:00010000:3.0:1789762764.984573:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762764.984662:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0327b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.984671:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0327b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.984679:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0327b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.984694:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03282 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.984701:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03282 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.984707:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03282 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762764.985448:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00010000:00010000:3.0:1789762764.985458:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0327b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.985466:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0327b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.985472:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0327b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.985477:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbfc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762764.985481:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0327b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.985487:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0327b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.985497:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46600) 00010000:00010000:3.0:1789762764.985500:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762764.985503:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0327b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.985509:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0327b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762764.985520:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007fe2e738 reply state 000000000544b8a0 transno 4294967467 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03282 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762764.985527:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762764.985586:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc02300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:13099:x1876702014931456:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 1088us (1190us total) trans 4294967467 rc 0/0 00000100:00100000:3.0:1789762764.985610:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 209 00010000:00010000:3.0:1789762764.986428:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03282 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.986437:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03282 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.986443:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03282 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.986448:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762764.986452:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03282 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.986457:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03282 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.986468:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e83a46200) 00010000:00010000:3.0:1789762764.986472:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762764.986475:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03282 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.986481:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03282 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762764.993003:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762764.993057:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014931584 00000100:00100000:0.0:1789762764.993066:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014931584 00000100:00100000:0.0:1789762764.993156:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 210 00000100:00100000:0.0:1789762764.993173:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:13100:x1876702014931584:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762764.993191:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762764.993207:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184800/0x416193e012b03289 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd480 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.993266:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185600/0x416193e012b03290 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.993277:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185600/0x416193e012b03290 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.993287:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185600/0x416193e012b03290 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762764.993310:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762764.993314:0:5805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762764.993328:0:5805:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185600/0x416193e012b03290 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.993347:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184800/0x416193e012b03289 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd480 expref: 28 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.993369:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185600/0x416193e012b03290 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd480 expref: 27 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.993377:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762764.993380:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762764.993383:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86185600, rc 0) 00000100:00100000:0.0:1789762764.993431:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:13100:x1876702014931584:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 265us (432us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762764.993439:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 210 00000100:00100000:2.0:1789762764.994687:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762764.994742:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014931840 00000100:00100000:0.0:1789762764.994749:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014931840 00000100:00100000:0.0:1789762764.994799:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 211 00000100:00100000:0.0:1789762764.994809:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4972a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:13100:x1876702014931840:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762764.994820:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762764.994822:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1789762764.994836:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185600/0x416193e012b03290 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd480 expref: 27 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.994848:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762764.994855:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762764.994865:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184c00/0x416193e012b03297 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd487 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.994955:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b0329e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.994965:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b0329e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.994973:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b0329e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.994986:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b032a5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.994992:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b032a5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.994999:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b032a5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995056:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b032ac lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995075:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b032ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995082:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b032ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762764.995146:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00010000:00010000:0.0:1789762764.995164:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b032ac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995175:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b032ac lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995184:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b032ac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995192:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762764.995196:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b032ac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995203:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b032ac lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995212:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762764.995214:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762764.995218:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b032ac lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995223:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b032ac lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995248:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b0329e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995262:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b0329e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995268:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b0329e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995273:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762764.995276:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b0329e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995282:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b0329e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995291:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3300) 00010000:00010000:0.0:1789762764.995293:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762764.995296:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b0329e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995302:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b0329e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995316:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b032a5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995322:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b032a5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995328:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b032a5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995332:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762764.995343:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b032a5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995349:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b032a5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995357:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3c00) 00010000:00010000:0.0:1789762764.995359:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762764.995361:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b032a5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995366:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b032a5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995393:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184c00/0x416193e012b03297 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd487 expref: 27 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995403:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3300) 00010000:00010000:0.0:1789762764.995405:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762764.995410:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184c00/0x416193e012b03297 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd487 expref: 27 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762764.995427:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86184c00, rc 0) 00000100:00100000:0.0:1789762764.995467:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4972a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:13100:x1876702014931840:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 660us (781us total) trans 0 rc 301/301 00000100:00100000:0.0:1789762764.995474:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 211 00000100:00100000:2.0:1789762764.996591:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014931968 00000100:00100000:2.0:1789762764.996598:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014931968 00000100:00100000:2.0:1789762764.996670:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 212 00000100:00100000:2.0:1789762764.996684:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:13100:x1876702014931968:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:2.0:1789762764.996697:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762764.996729:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b032b3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd48e expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762764.996778:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b032ba lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762764.996790:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b032ba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762764.996801:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b032ba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762764.996823:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762764.996826:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762764.996840:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b032ba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762764.996861:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b032b3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd48e expref: 28 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762764.996880:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b032ba lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd48e expref: 27 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762764.996888:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762764.996892:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762764.996896:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa33e00, rc 0) 00000100:00100000:2.0:1789762764.996946:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:13100:x1876702014931968:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 265us (400us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762764.996954:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 212 00000100:00100000:2.0:1789762764.998157:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762764.998206:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014932224 00000100:00100000:3.0:1789762764.998214:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014932224 00000100:00100000:3.0:1789762764.998249:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 213 00000100:00100000:3.0:1789762764.998259:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4972300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:13100:x1876702014932224:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762764.998269:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762764.998289:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b032c1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd495 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998325:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032c8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998334:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032c8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998341:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032c8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998353:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032cf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998359:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032cf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998364:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032cf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762764.998415:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00010000:00010000:3.0:1789762764.998503:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032c8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998514:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032c8 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998522:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032c8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998531:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbfc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762764.998536:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032c8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998552:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032c8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998565:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:3.0:1789762764.998569:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762764.998574:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032c8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998582:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032c8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998596:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032cf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998605:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032cf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998613:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032cf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998639:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762764.998643:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032cf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998649:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032cf lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998657:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e83a46200) 00010000:00010000:3.0:1789762764.998659:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762764.998661:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032cf lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998678:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032cf lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998709:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b032c1 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd495 expref: 29 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998718:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:3.0:1789762764.998720:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762764.998724:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b032c1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd495 expref: 29 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762764.998732:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbf400, rc 0) 00000100:00100000:3.0:1789762764.998771:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4972300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:13100:x1876702014932224:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 514us (617us total) trans 4294967468 rc 301/301 00000100:00100000:3.0:1789762764.998778:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 213 00000100:00100000:2.0:1789762764.999851:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762764.999895:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014932480 00000100:00100000:3.0:1789762764.999902:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014932480 00000100:00100000:3.0:1789762764.999959:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 214 00000100:00100000:3.0:1789762764.999970:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:13100:x1876702014932480:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:3.0:1789762765.000033:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b032dd lrc: 3/0,1 mode: --/PW res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.000050:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b032dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.000064:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b032dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762765.000285:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000889ed527 reply state 0000000084b510dd transno 4294967469 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b032dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762765.000294:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762765.000300:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00000100:00100000:3.0:1789762765.000349:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:13100:x1876702014932480:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 381us (498us total) trans 4294967469 rc 0/0 00000100:00100000:3.0:1789762765.000356:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 214 00010000:00010000:3.0:1789762765.001283:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b032dd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.001292:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b032dd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.001298:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b032dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.001303:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762765.001305:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b032dd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.001311:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b032dd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.001319:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981e83a46600) 00010000:00010000:3.0:1789762765.001322:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789762765.001324:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b032dd lrc: 2/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.001329:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b032dd lrc: 0/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762765.001500:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762765.001644:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762765.006360:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.006421:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014932864 00000100:00100000:3.0:1789762765.006430:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014932864 00000100:00100000:3.0:1789762765.006484:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 215 00000100:00100000:3.0:1789762765.006501:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:13101:x1876702014932864:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:3.0:1789762765.006520:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762765.006538:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b032e4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd49c expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006581:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032eb lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006592:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032eb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006601:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032eb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006613:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032f2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006638:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032f2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006651:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032f2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006715:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b032f9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006723:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b032f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006740:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b032f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762765.006763:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00000004:00200000:3.0:1789762765.006766:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762765.006779:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b032f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006788:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032eb lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006794:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032eb lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006799:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032eb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006805:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762765.006808:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032eb lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006814:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032eb lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006823:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:3.0:1789762765.006826:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762765.006829:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032eb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006834:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b032eb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006857:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032f2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006863:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032f2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006868:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032f2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006873:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbfc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762765.006875:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032f2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006881:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032f2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006887:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e83a46200) 00010000:00010000:3.0:1789762765.006890:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762765.006892:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032f2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006897:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b032f2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006923:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b032e4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd49c expref: 32 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006937:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b032f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd49c expref: 31 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.006956:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981e83a46000) 00010000:00010000:3.0:1789762765.006959:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762765.006961:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbe000, rc 0) 00000100:00100000:3.0:1789762765.007007:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13101:x1876702014932864:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 513us (650us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762765.007014:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 215 00000100:00100000:2.0:1789762765.008279:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.008318:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014933120 00000100:00100000:3.0:1789762765.008324:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014933120 00000100:00100000:3.0:1789762765.008357:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 216 00000100:00100000:3.0:1789762765.008365:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13101:x1876702014933120:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:3.0:1789762765.008389:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762765.008401:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b032f9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd49c expref: 31 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.008411:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981e83a46000) 00010000:00010000:3.0:1789762765.008434:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762765.008453:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03300 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.008466:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03300 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.008478:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03300 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762765.008639:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e746e4e0 reply state 00000000c880fb18 transno 4294967471 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03300 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762765.008661:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762765.008668:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00000100:00100000:3.0:1789762765.008720:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:13101:x1876702014933120:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 355us (443us total) trans 4294967471 rc 0/0 00000100:00100000:3.0:1789762765.008727:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 216 00010000:00010000:3.0:1789762765.009418:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03300 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.009431:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03300 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.009439:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03300 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.009447:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762765.009453:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03300 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.009462:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03300 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.009474:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981e83a46600) 00010000:00010000:3.0:1789762765.009478:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762765.009481:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03300 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.009487:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03300 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762765.029854:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014933248 00000100:00100000:3.0:1789762765.029873:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014933248 00000100:00100000:3.0:1789762765.029944:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 217 00000100:00100000:3.0:1789762765.029959:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad02a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13107:x1876702014933248:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762765.029983:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762765.030004:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03307 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4a3 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030083:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0330e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030141:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0330e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030160:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0330e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030181:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03315 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030192:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03315 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030203:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03315 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030329:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0331c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030338:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0331c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030345:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0331c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762765.030378:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00000004:00200000:3.0:1789762765.030381:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762765.030399:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0331c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030410:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0330e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030416:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0330e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030422:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0330e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030428:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762765.030432:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0330e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030438:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0330e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030450:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:3.0:1789762765.030453:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762765.030457:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0330e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030462:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0330e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030472:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03315 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030486:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03315 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030492:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03315 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030497:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762765.030500:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03315 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030505:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03315 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030513:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e83a46600) 00010000:00010000:3.0:1789762765.030516:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762765.030518:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03315 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030522:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b03315 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030556:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b03307 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4a3 expref: 33 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030571:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0331c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4a3 expref: 32 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.030577:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981e83a46000) 00010000:00010000:3.0:1789762765.030580:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789762765.030587:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbfc00, rc 0) 00000100:00100000:3.0:1789762765.030660:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad02a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:13107:x1876702014933248:12345-192.168.203.48@tcp:101:dd.0 Request processed in 706us (942us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762765.030668:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 217 00000100:00100000:2.0:1789762765.031837:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762765.031915:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014933504 00000100:00100000:0.0:1789762765.031925:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014933504 00000100:00100000:0.0:1789762765.031980:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 218 00000100:00100000:0.0:1789762765.032005:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:13107:x1876702014933504:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762765.032020:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762765.032035:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184c00/0x416193e012b03323 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4aa expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032149:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b0332a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032160:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b0332a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032168:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b0332a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032178:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03331 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032183:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03331 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032189:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03331 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762765.032584:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xe:0x0], index 0: 0 00010000:00010000:0.0:1789762765.032760:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b03338 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x24:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032769:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b03338 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032776:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b03338 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032811:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b0332a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032817:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b0332a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032822:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b0332a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032826:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762765.032829:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b0332a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032834:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b0332a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032842:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981e83a46000) 00010000:00010000:0.0:1789762765.032845:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762765.032848:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b0332a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032853:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185a00/0x416193e012b0332a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762765.032872:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000024cdc7fe reply state 0000000080696092 transno 4294967472 ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03331 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762765.032883:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762765.032903:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184c00/0x416193e012b03323 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4aa expref: 35 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032926:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b03338 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4aa expref: 34 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762765.032933:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x24:0x0].0x0 (ffff981ea0aa3c00) 00010000:00010000:0.0:1789762765.032935:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762765.032937:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86185e00, rc 0) 00000100:00100000:0.0:1789762765.032974:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:13107:x1876702014933504:12345-192.168.203.48@tcp:101:dd.0 Request processed in 975us (1139us total) trans 4294967472 rc 0/0 00000100:00100000:0.0:1789762765.032980:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 218 00010000:00010000:3.0:1789762765.033692:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03331 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.033701:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03331 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.033707:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03331 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.033712:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762765.033715:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03331 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.033720:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03331 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.033729:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4009f790 (ffff981ea0aa3300) 00010000:00010000:3.0:1789762765.033748:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762765.033751:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03331 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.033756:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03331 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762765.037057:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.037217:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014933632 00000100:00100000:3.0:1789762765.037225:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014933632 00000100:00080000:2.0:1789762765.037265:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762765.037282:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287829086285 time 287829198235 state 0x0 ref 1 00000100:00100000:3.0:1789762765.037319:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 287879198235 time 287829198235 state 0x1 ref 1 00000100:00100000:3.0:1789762765.037343:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 2 deadline 287879198235 time 288341251525 state 0x0 ref 1 00000100:00100000:3.0:1789762765.037353:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2364:x1876702014933632:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762765.037385:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762765.037391:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03346 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.037403:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=288s v=(0 0) 00010000:00010000:3.0:1789762765.037407:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762765.037411:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.037438:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762765.070192:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:3.0:1789762765.070199:0:11723:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb mtime from disk: 0 -> 1789762765 00002000:00010000:3.0:1789762765.070203:0:11723:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb ctime from disk: 0 -> 1789762765 00002000:00010000:3.0:1789762765.070207:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:3.0:1789762765.070222:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03346 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.070241:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03346 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.070263:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03346 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.070270:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762765.070274:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03346 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.070282:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03346 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.070299:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03346 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.070306:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03346 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762765.070391:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014933632:12345-192.168.203.48@tcp:4:dd.0 Request processed in 33035us (33331us total) trans 4294967690 rc 0/0 00000100:00100000:3.0:1789762765.070401:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 261 00000100:00100000:3.0:1789762765.070412:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 2 deadline 287879198235 time 288341251525 state 0x2 ref 0 00000100:00100000:2.0:1789762765.086041:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.086110:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014933760 00000100:00100000:3.0:1789762765.086120:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014933760 00000100:00100000:3.0:1789762765.086166:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 2 deadline 287879198235 time 288341251525 state 0x0 ref 1 00000100:00100000:3.0:1789762765.086180:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 2 deadline 288391251525 time 288341251525 state 0x1 ref 1 00000100:00100000:3.0:1789762765.086222:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 1 deadline 288391251525 time 288390126681 state 0x0 ref 1 00000100:00100000:3.0:1789762765.086239:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014933760:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762765.086281:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 1048576, mtime: 0x6aad9ccd, atime: 0x0, ctime: 0x6aad9ccd, blocks: 0x803 00010000:00010000:3.0:1789762765.086290:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b0334d lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.086303:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=288s v=(0 0) 00010000:00010000:3.0:1789762765.086308:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762765.086313:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b0334d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.086324:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b0334d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762765.101976:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967691 is committed 00002000:00010000:3.0:1789762765.101990:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:3.0:1789762765.101997:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:3.0:1789762765.102009:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b0334d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.102021:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b0334d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.102028:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b0334d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.102046:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762765.102051:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b0334d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.102059:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b0334d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.102077:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b0334d lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.102119:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b0334d lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762765.102196:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014933760:12345-192.168.203.48@tcp:4:dd.0 Request processed in 15958us (16157us total) trans 4294967691 rc 0/0 00000100:00100000:3.0:1789762765.102206:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 262 00000100:00100000:3.0:1789762765.102217:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 1 deadline 288391251525 time 288390126681 state 0x2 ref 0 00000100:00100000:2.0:1789762765.104651:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762765.119466:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967692 is committed 00000100:00100000:0.0:1789762765.122707:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762765.165799:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967693 is committed 00000100:00100000:2.0:1789762765.168750:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.168813:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014934144 00000100:00100000:3.0:1789762765.168824:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014934144 00000100:00100000:3.0:1789762765.168871:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288458734985 time 288458793443 state 0x0 ref 1 00000100:00100000:3.0:1789762765.168888:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288508793443 time 288458793443 state 0x1 ref 1 00000001:00080000:0.0:1789762765.218770:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967694 is committed 00002000:00010000:3.0:1789762765.218782:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:3.0:1789762765.218794:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:3.0:1789762765.218807:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b03362 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.218816:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b03362 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.218823:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b03362 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.218828:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762765.218831:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b03362 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.218837:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b03362 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.218852:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b03362 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.218857:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b03362 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762765.218931:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014934144:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13924us (50181us total) trans 4294967694 rc 0/0 00000100:00100000:3.0:1789762765.218942:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 265 00000100:00100000:3.0:1789762765.218954:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288508793443 time 288508885692 state 0x2 ref 0 00000100:00100000:0.0:1789762765.221931:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762765.269122:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967695 is committed 00000100:00100000:2.0:1789762765.272176:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.272220:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014934400 00000100:00100000:3.0:1789762765.272228:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014934400 00000100:00100000:3.0:1789762765.272263:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288558885692 time 288559041028 state 0x0 ref 1 00000100:00100000:3.0:1789762765.272274:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288609041028 time 288559041028 state 0x1 ref 1 00000100:00100000:3.0:1789762765.305201:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288609041028 time 288609094525 state 0x0 ref 1 00000100:00100000:3.0:1789762765.305230:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014934400:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762765.305278:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 6291456, mtime: 0x6aad9ccd, atime: 0x0, ctime: 0x6aad9ccd, blocks: 0x3007 00010000:00010000:3.0:1789762765.305285:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03370 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.305299:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=288s v=(0 0) 00010000:00010000:3.0:1789762765.305303:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762765.305308:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03370 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762765.305319:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b03370 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762765.319795:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967696 is committed 00000100:00100000:0.0:1789762765.323009:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:3.0:1789762765.372778:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967697 is committed 00000100:00100000:2.0:1789762765.376223:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.376279:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014934656 00000100:00100000:3.0:1789762765.376290:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014934656 00000100:00100000:3.0:1789762765.376349:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288659094525 time 288659190929 state 0x0 ref 1 00000100:00100000:3.0:1789762765.376366:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288709190929 time 288659190929 state 0x1 ref 1 00000001:00080000:3.0:1789762765.418857:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967698 is committed 00000100:00100000:0.0:1789762765.421781:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762765.470412:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967699 is committed 00000100:00100000:2.0:1789762765.473066:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.473111:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014934912 00000100:00100000:3.0:1789762765.473120:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014934912 00000100:00100000:3.0:1789762765.473167:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288759284890 time 288759355238 state 0x0 ref 1 00000100:00100000:3.0:1789762765.473178:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288809355238 time 288759355238 state 0x1 ref 1 00000001:00080000:3.0:1789762765.519279:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967700 is committed 00000100:00100000:0.0:1789762765.522329:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:3.0:1789762765.571079:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967701 is committed 00000100:00100000:2.0:1789762765.574850:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.574920:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014935168 00000100:00100000:3.0:1789762765.574933:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014935168 00000100:00100000:3.0:1789762765.574988:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288859467374 time 288859560795 state 0x0 ref 1 00000100:00100000:3.0:1789762765.575005:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288909560795 time 288859560795 state 0x1 ref 1 00000001:00080000:2.0:1789762765.622698:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967702 is committed 00000100:00100000:0.0:1789762765.626182:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762765.676977:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967703 is committed 00000100:00100000:0.0:1789762765.681685:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.681766:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014935424 00000100:00100000:3.0:1789762765.681781:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014935424 00000100:00100000:3.0:1789762765.681845:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 288959727726 time 288959809276 state 0x0 ref 1 00000100:00100000:3.0:1789762765.681864:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289009809276 time 288959809276 state 0x1 ref 1 00000001:00080000:2.0:1789762765.728876:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967704 is committed 00000100:00100000:2.0:1789762765.733725:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762765.773201:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967705 is committed 00000100:00100000:0.0:1789762765.776518:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.776576:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014935680 00000100:00100000:3.0:1789762765.776586:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014935680 00000100:00100000:3.0:1789762765.776659:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289059982012 time 289060038182 state 0x0 ref 1 00000100:00100000:3.0:1789762765.776673:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289110038182 time 289060038182 state 0x1 ref 1 00000001:00080000:2.0:1789762765.818744:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967706 is committed 00000100:00100000:2.0:1789762765.821574:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762765.868135:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967707 is committed 00000100:00100000:2.0:1789762765.870705:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.870746:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014935936 00000100:00100000:3.0:1789762765.870755:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014935936 00000100:00100000:3.0:1789762765.870799:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289160156391 time 289160209373 state 0x0 ref 1 00000100:00100000:3.0:1789762765.870812:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289210209373 time 289160209373 state 0x1 ref 1 00000001:00080000:3.0:1789762765.920239:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967708 is committed 00000100:00100000:0.0:1789762765.922901:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:3.0:1789762765.967905:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967709 is committed 00000100:00100000:2.0:1789762765.970384:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762765.970428:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014936192 00000100:00100000:3.0:1789762765.970436:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014936192 00000100:00100000:3.0:1789762765.970473:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289260339977 time 289260405827 state 0x0 ref 1 00000100:00100000:3.0:1789762765.970484:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289310405827 time 289260405827 state 0x1 ref 1 00000001:00080000:2.0:1789762766.020051:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967710 is committed 00000100:00100000:2.0:1789762766.022730:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:3.0:1789762766.069140:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967711 is committed 00000100:00100000:3.0:1789762766.071507:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014936448 00000100:00100000:3.0:1789762766.071515:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014936448 00000100:00100000:3.0:1789762766.071552:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289360482642 time 289360549915 state 0x0 ref 1 00000100:00100000:3.0:1789762766.071563:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289410549915 time 289360549915 state 0x1 ref 1 00000001:00080000:0.0:1789762766.119778:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967712 is committed 00002000:00010000:3.0:1789762766.119782:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:3.0:1789762766.119790:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:3.0:1789762766.119812:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b033e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.119827:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b033e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.119836:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b033e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.119842:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762766.119846:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b033e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.119853:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b033e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.119869:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b033e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.119875:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b033e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762766.119948:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014936448:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13178us (48483us total) trans 4294967712 rc 0/0 00000100:00100000:3.0:1789762766.119959:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 283 00000100:00100000:3.0:1789762766.119972:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289410549915 time 289410647894 state 0x2 ref 0 00000100:00100000:0.0:1789762766.122348:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762766.173087:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967713 is committed 00000100:00100000:2.0:1789762766.176887:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762766.176938:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014936704 00000100:00100000:3.0:1789762766.176948:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014936704 00000100:00100000:3.0:1789762766.176994:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289460647894 time 289460724748 state 0x0 ref 1 00000100:00100000:3.0:1789762766.177008:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289510724748 time 289460724748 state 0x1 ref 1 00000100:00100000:3.0:1789762766.206896:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289510724748 time 289510787892 state 0x0 ref 1 00000100:00100000:3.0:1789762766.206923:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014936704:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762766.206983:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 25165824, mtime: 0x6aad9cce, atime: 0x0, ctime: 0x6aad9cce, blocks: 0xc007 00010000:00010000:3.0:1789762766.206993:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000078bd4209/0x416193e012b033ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.207020:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=289s v=(0 0) 00010000:00010000:3.0:1789762766.207028:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762766.207036:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078bd4209/0x416193e012b033ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.207056:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078bd4209/0x416193e012b033ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762766.222994:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967714 is committed 00000100:00100000:0.0:1789762766.227068:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762766.271947:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967715 is committed 00000100:00100000:3.0:1789762766.275235:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762766.275287:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014936960 00000100:00100000:0.0:1789762766.275301:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014936960 00000100:00100000:0.0:1789762766.275338:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289560787892 time 289560859888 state 0x0 ref 1 00000100:00100000:0.0:1789762766.275350:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289610859888 time 289560859888 state 0x1 ref 1 00000001:00080000:2.0:1789762766.322278:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967716 is committed 00000100:00100000:0.0:1789762766.325432:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762766.371110:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967717 is committed 00000100:00100000:0.0:1789762766.374253:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762766.374301:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014937216 00000100:00100000:3.0:1789762766.374310:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014937216 00000100:00100000:3.0:1789762766.374346:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289661029183 time 289661131743 state 0x0 ref 1 00000100:00100000:3.0:1789762766.374364:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289711131743 time 289661131743 state 0x1 ref 1 00000001:00080000:3.0:1789762766.426859:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967718 is committed 00000100:00100000:3.0:1789762766.431282:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762766.457504:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289761211843 time 289761388409 state 0x0 ref 1 00002000:00010000:3.0:1789762766.458490:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 30408704, mtime: 0x6aad9cce, atime: 0x0, ctime: 0x6aad9cce, blocks: 0xe807 00010000:00010000:3.0:1789762766.458507:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b03411 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.458529:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=289s v=(0 0) 00010000:00010000:3.0:1789762766.458539:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762766.458549:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b03411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.458570:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b03411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762766.481809:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022156544:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789762766.481934:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad03800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022156672:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1789762766.481935:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762766.482026:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762766.482217:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:3.0:1789762766.482247:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb494e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022156544:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1789762766.482293:0:10916:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:8 count 1 00002000:00010000:3.0:1789762766.482353:0:10916:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x8:0x0] initial LVB size: 8388608, mtime: 0x6aad9ccb, atime: 0x0, ctime: 0x6aad9ccb, blocks: 0x803 00010000:00010000:3.0:1789762766.482372:0:10916:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c187a00f/0x416193e012b03418 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.482474:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x8:0x0].0x0 since=289s v=(0 0) 00010000:00010000:3.0:1789762766.482490:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x8:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762766.482515:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c187a00f/0x416193e012b03418 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.482557:0:10916:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c187a00f/0x416193e012b03418 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.482584:0:10916:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c187a00f/0x416193e012b03418 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.482760:0:10916:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c187a00f/0x416193e012b03418 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.482819:0:10916:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762766.482869:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762766.482885:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c187a00f/0x416193e012b03418 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.482910:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762766.482923:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c187a00f/0x416193e012b03418 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.482946:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c187a00f/0x416193e012b03418 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.482983:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c187a00f/0x416193e012b03418 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.482999:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c187a00f/0x416193e012b03418 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x8:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.483040:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762766.483046:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762766.483050:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1789762766.484598:0:3274:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981d8257fb80/2, rc 0, transno 4294967379 00000100:00100000:0.0:1789762766.484647:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022156544:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789762766.484747:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb494e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022156544:12345-0@lo:6:osp-syn-1-0.0 Request processed in 2508us (2817us total) trans 4294967379 rc 0/0 00000100:00100000:3.0:1789762766.484767:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00100000:0.0:1789762766.484858:0:6647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad03480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6647:1876702022156800:192.168.203.48@tcp:104: 00010000:00010000:0.0:1789762766.484907:0:6647:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 3/0,1 mode: --/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762766.485319:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762766.485504:0:8405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014937472 00000100:00100000:3.0:1789762766.485516:0:8405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014937472 00010000:00010000:3.0:1789762766.485564:0:8405:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b031e8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd45d expref: 10 pid: 6649 timeout: 389 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762766.485639:0:8405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 58 00000100:00100000:3.0:1789762766.485655:0:8405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86f03050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014937472:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789762766.485687:0:8405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762766.485697:0:8405:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b031e8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd45d expref: 10 pid: 6649 timeout: 389 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.485723:0:8405:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b031e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd45d expref: 10 pid: 6649 timeout: 389 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.485751:0:8405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b031e8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd45d expref: 10 pid: 6649 timeout: 389 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.485773:0:8405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762766.485779:0:8405:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762766.485786:0:8405:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.485799:0:8405:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.485836:0:8405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762766.485870:0:6647:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.485888:0:6647:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762766.485893:0:8405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86f03050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702014937472:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 239us (582us total) trans 0 rc 0/0 00010000:00010000:0.0:1789762766.485926:0:6647:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762766.485927:0:8405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 58 00010000:00010000:0.0:1789762766.485977:0:6647:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.486024:0:6647:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762766.486072:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762766.486133:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.486169:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762766.486176:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.486190:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.486212:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 1/0,0 mode: --/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.486223:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0341f lrc: 0/0,0 mode: --/PW res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.486260:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762766.486265:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762766.486272:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762766.487244:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022156672:12345-0@lo:6:osp-syn-0-0.0 Request processed in 4933us (5223us total) trans 4294967720 rc 0/0 00000100:00100000:0.0:1789762766.487260:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00002000:00010000:3.0:1789762766.487748:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 30408704 -> 31457280 00000001:00080000:0.0:1789762766.487750:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967719 is committed 00002000:00010000:3.0:1789762766.487761:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 59399 -> 61447 00010000:00010000:3.0:1789762766.487777:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b03411 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.487815:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b03411 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.487826:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b03411 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.487834:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762766.487840:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b03411 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.487851:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b03411 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.487868:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b03411 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.487877:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b03411 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762766.487957:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014937344:12345-192.168.203.48@tcp:4:dd.0 Request processed in 30424us (56676us total) trans 4294967719 rc 0/0 00000100:00100000:3.0:1789762766.487973:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 290 00000100:00100000:3.0:1789762766.487990:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289761211843 time 289761388409 state 0x2 ref 0 00000100:00100000:2.0:1789762766.491668:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762766.491750:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014937600 00000100:00100000:3.0:1789762766.491772:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014937600 00000100:00100000:3.0:1789762766.491821:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289761211843 time 289761388409 state 0x0 ref 1 00000100:00100000:3.0:1789762766.491839:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289811388409 time 289761388409 state 0x1 ref 1 00000001:00080000:0.0:1789762766.528850:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967720 is committed 00000001:00080000:0.0:1789762766.528875:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967721 is committed 00002000:00010000:2.0:1789762766.528893:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:2.0:1789762766.528904:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 61447 -> 63495 00010000:00010000:2.0:1789762766.528922:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b03426 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.528940:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b03426 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.528952:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b03426 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.528976:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762766.528982:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b03426 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.528994:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b03426 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.529018:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b03426 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.529029:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b03426 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762766.529229:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014937600:12345-192.168.203.48@tcp:4:dd.0 Request processed in 21523us (37555us total) trans 4294967721 rc 0/0 00000100:00100000:2.0:1789762766.529250:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 291 00000100:00100000:2.0:1789762766.529273:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289811388409 time 289811578787 state 0x2 ref 0 00000100:00100000:0.0:1789762766.533350:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762766.557817:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289861578787 time 289861711629 state 0x0 ref 1 00000100:00100000:3.0:1789762766.557844:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014937728:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762766.557901:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 32505856, mtime: 0x6aad9cce, atime: 0x0, ctime: 0x6aad9cce, blocks: 0xf807 00010000:00010000:3.0:1789762766.557913:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0342d lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.557931:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=289s v=(0 0) 00010000:00010000:3.0:1789762766.557937:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762766.557945:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0342d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.557962:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0342d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762766.580445:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:3.0:1789762766.580456:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 63495 -> 65543 00000001:00080000:0.0:1789762766.580459:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967722 is committed 00010000:00010000:3.0:1789762766.580501:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0342d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.580520:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0342d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.580542:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0342d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.580552:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762766.580557:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0342d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.580568:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0342d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.580589:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0342d lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.580598:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0342d lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762766.580726:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014937728:12345-192.168.203.48@tcp:4:dd.0 Request processed in 22884us (47373us total) trans 4294967722 rc 0/0 00000100:00100000:3.0:1789762766.580745:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 292 00000100:00100000:3.0:1789762766.580764:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289861578787 time 289861711629 state 0x2 ref 0 00000100:00100000:3.0:1789762766.584385:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014937856 00000100:00100000:3.0:1789762766.584395:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014937856 00000100:00100000:3.0:1789762766.584446:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289861578787 time 289861711629 state 0x0 ref 1 00000100:00100000:3.0:1789762766.584476:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289911711629 time 289861711629 state 0x1 ref 1 00000100:00100000:2.0:1789762766.607971:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289911711629 time 289911865221 state 0x0 ref 1 00000100:00100000:2.0:1789762766.607999:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014937856:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:2.0:1789762766.608058:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 33554432, mtime: 0x6aad9cce, atime: 0x0, ctime: 0x6aad9cce, blocks: 0x10007 00010000:00010000:2.0:1789762766.608071:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03434 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.608137:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=289s v=(0 0) 00010000:00010000:2.0:1789762766.608143:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762766.608150:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.608166:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762766.627155:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:2.0:1789762766.627165:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 65543 -> 67591 00010000:00010000:2.0:1789762766.627183:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03434 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762766.627193:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967723 is committed 00010000:00010000:2.0:1789762766.627203:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03434 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.627218:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03434 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.627228:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762766.627234:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03434 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.627246:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03434 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.627285:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03434 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.627295:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03434 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762766.627411:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014937856:12345-192.168.203.48@tcp:4:dd.0 Request processed in 19414us (43075us total) trans 4294967723 rc 0/0 00000100:00100000:2.0:1789762766.627424:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 293 00000100:00100000:2.0:1789762766.627439:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289911711629 time 289911865221 state 0x2 ref 0 00000100:00100000:2.0:1789762766.631489:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762766.658170:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289961865221 time 289962066552 state 0x0 ref 1 00000100:00100000:3.0:1789762766.658199:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014937984:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762766.658248:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 34603008, mtime: 0x6aad9cce, atime: 0x0, ctime: 0x6aad9cce, blocks: 0x10807 00010000:00010000:3.0:1789762766.658259:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0343b lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.658278:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=289s v=(0 0) 00010000:00010000:3.0:1789762766.658284:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762766.658291:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0343b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.658326:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0343b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762766.680269:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:2.0:1789762766.680277:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 67591 -> 69639 00010000:00010000:2.0:1789762766.680292:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0343b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.680308:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0343b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.680319:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0343b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.680328:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762766.680334:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0343b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.680375:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0343b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.680401:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0343b lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.680410:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b0343b lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762766.680516:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014937984:12345-192.168.203.48@tcp:4:dd.0 Request processed in 22322us (49026us total) trans 4294967724 rc 0/0 00000100:00100000:2.0:1789762766.680531:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 294 00000100:00100000:2.0:1789762766.680546:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289961865221 time 289962066552 state 0x2 ref 0 00000100:00100000:3.0:1789762766.684734:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014938112 00000100:00100000:3.0:1789762766.684747:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014938112 00000100:00100000:3.0:1789762766.684802:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 289961865221 time 289962066552 state 0x0 ref 1 00000100:00100000:3.0:1789762766.684819:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290012066552 time 289962066552 state 0x1 ref 1 00000100:00100000:2.0:1789762766.708353:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290012066552 time 290012250440 state 0x0 ref 1 00000100:00100000:2.0:1789762766.708396:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014938112:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:2.0:1789762766.708451:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 35651584, mtime: 0x6aad9cce, atime: 0x0, ctime: 0x6aad9cce, blocks: 0x11007 00010000:00010000:2.0:1789762766.708463:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03442 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.708482:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=290s v=(0 0) 00010000:00010000:2.0:1789762766.708488:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762766.708496:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03442 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.708530:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03442 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762766.735513:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014938240 00000100:00100000:0.0:1789762766.735527:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014938240 00000100:00100000:0.0:1789762766.735599:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290012066552 time 290012250440 state 0x0 ref 1 00000100:00100000:0.0:1789762766.735669:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290062250440 time 290012250440 state 0x1 ref 1 00000100:00100000:2.0:1789762766.758520:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290062250440 time 290062411937 state 0x0 ref 1 00000100:00100000:2.0:1789762766.758567:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014938240:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:2.0:1789762766.758666:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 36700160, mtime: 0x6aad9cce, atime: 0x0, ctime: 0x6aad9cce, blocks: 0x11807 00010000:00010000:2.0:1789762766.758678:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03449 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.758698:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=290s v=(0 0) 00010000:00010000:2.0:1789762766.758704:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762766.758712:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.758729:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.775311:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789762766.783907:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 36700160 -> 37748736 00000001:00080000:3.0:1789762766.783915:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967726 is committed 00002000:00010000:2.0:1789762766.783917:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 71687 -> 73735 00010000:00010000:2.0:1789762766.783935:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03449 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.783954:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03449 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.783982:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03449 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.783993:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762766.783999:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03449 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.784011:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03449 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.784033:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03449 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.784043:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03449 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762766.784225:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014938240:12345-192.168.203.48@tcp:4:dd.0 Request processed in 25679us (48811us total) trans 4294967726 rc 0/0 00000100:00100000:2.0:1789762766.784239:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 296 00000100:00100000:2.0:1789762766.784252:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290062250440 time 290062411937 state 0x2 ref 0 00000100:00100000:2.0:1789762766.788276:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762766.788350:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014938368 00000100:00100000:3.0:1789762766.788365:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014938368 00000100:00100000:3.0:1789762766.788433:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290062250440 time 290062411937 state 0x0 ref 1 00000100:00100000:3.0:1789762766.788452:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290112411937 time 290062411937 state 0x1 ref 1 00000001:00080000:2.0:1789762766.829567:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967727 is committed 00002000:00010000:0.0:1789762766.829706:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:0.0:1789762766.829718:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 73735 -> 75783 00010000:00010000:0.0:1789762766.829738:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b03450 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.829761:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b03450 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.829788:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b03450 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.829797:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762766.829804:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b03450 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.829815:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b03450 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.829844:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b03450 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.829858:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b03450 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762766.829986:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014938368:12345-192.168.203.48@tcp:4:dd.0 Request processed in 21196us (41704us total) trans 4294967727 rc 0/0 00000100:00100000:0.0:1789762766.830009:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 297 00000100:00100000:0.0:1789762766.830028:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290112411937 time 290112644216 state 0x2 ref 0 00000100:00100000:2.0:1789762766.834461:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762766.834517:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014938496 00000100:00100000:0.0:1789762766.834529:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014938496 00000100:00100000:0.0:1789762766.834580:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290112411937 time 290112644216 state 0x0 ref 1 00000100:00100000:0.0:1789762766.834640:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290162644216 time 290112644216 state 0x1 ref 1 00000100:00100000:2.0:1789762766.858866:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290162644216 time 290162759263 state 0x0 ref 1 00000100:00100000:2.0:1789762766.858910:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014938496:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:2.0:1789762766.858967:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 38797312, mtime: 0x6aad9cce, atime: 0x0, ctime: 0x6aad9cce, blocks: 0x12807 00010000:00010000:2.0:1789762766.858981:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8724713/0x416193e012b03457 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.859004:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=290s v=(0 0) 00010000:00010000:2.0:1789762766.859011:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762766.859022:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8724713/0x416193e012b03457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762766.859043:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8724713/0x416193e012b03457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762766.881715:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:0.0:1789762766.881726:0:6654:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb mtime from disk: 1789762766 -> 1789762767 00002000:00010000:0.0:1789762766.881732:0:6654:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb ctime from disk: 1789762766 -> 1789762767 00002000:00010000:0.0:1789762766.881737:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 75783 -> 77831 00010000:00010000:0.0:1789762766.881759:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f8724713/0x416193e012b03457 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.881786:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f8724713/0x416193e012b03457 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.881806:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f8724713/0x416193e012b03457 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.881818:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762766.881825:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8724713/0x416193e012b03457 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.881854:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f8724713/0x416193e012b03457 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.881879:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f8724713/0x416193e012b03457 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.881888:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8724713/0x416193e012b03457 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762766.882029:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014938496:12345-192.168.203.48@tcp:4:dd.0 Request processed in 23121us (47564us total) trans 4294967728 rc 0/0 00000100:00100000:0.0:1789762766.882043:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 298 00000100:00100000:0.0:1789762766.882060:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290162644216 time 290162759263 state 0x2 ref 0 00000100:00100000:0.0:1789762766.886860:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762766.886968:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014938624 00000100:00100000:3.0:1789762766.886982:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014938624 00000100:00100000:3.0:1789762766.887046:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290162644216 time 290162759263 state 0x0 ref 1 00000100:00100000:3.0:1789762766.887066:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290212759263 time 290162759263 state 0x1 ref 1 00000100:00100000:0.0:1789762766.908987:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290212759263 time 290212877616 state 0x0 ref 1 00000100:00100000:0.0:1789762766.909026:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014938624:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762766.909151:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 39845888, mtime: 0x6aad9ccf, atime: 0x0, ctime: 0x6aad9ccf, blocks: 0x13007 00010000:00010000:0.0:1789762766.909164:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0345e lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.909184:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=290s v=(0 0) 00010000:00010000:0.0:1789762766.909190:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762766.909216:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0345e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.909256:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0345e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762766.930379:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 39845888 -> 40894464 00002000:00010000:0.0:1789762766.930389:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 77831 -> 79879 00010000:00010000:0.0:1789762766.930405:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0345e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.930435:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0345e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.930447:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0345e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.930456:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762766.930461:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0345e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.930472:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0345e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.930494:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0345e lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.930528:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0345e lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762766.930639:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014938624:12345-192.168.203.48@tcp:4:dd.0 Request processed in 21624us (43777us total) trans 4294967729 rc 0/0 00000100:00100000:0.0:1789762766.930651:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 299 00000100:00100000:0.0:1789762766.930662:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290212759263 time 290212877616 state 0x2 ref 0 00000100:00100000:0.0:1789762766.934714:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014938752 00000100:00100000:0.0:1789762766.934733:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014938752 00000100:00100000:0.0:1789762766.934780:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290212759263 time 290212877616 state 0x0 ref 1 00000100:00100000:0.0:1789762766.934795:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290262877616 time 290212877616 state 0x1 ref 1 00000100:00100000:2.0:1789762766.946539:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762766.946572:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762766.946596:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762766.946658:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762766.946659:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014938880 00000100:00100000:3.0:1789762766.946668:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014938880 00000100:00100000:0.0:1789762766.946686:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014939008 00000100:00100000:0.0:1789762766.946695:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014939008 00000100:00100000:2.0:1789762766.946695:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014939264 00000100:00100000:2.0:1789762766.946707:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014939264 00000100:00100000:3.0:1789762766.946732:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 286 00000100:00100000:3.0:1789762766.946765:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014938880:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762766.946780:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 203 00000100:00100000:0.0:1789762766.946801:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014939008:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762766.946826:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 204 00000100:00100000:2.0:1789762766.946850:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014939264:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762766.946870:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014938880:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 116us (322us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762766.946887:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014939008:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 87us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762766.946902:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 286 00000100:00100000:0.0:1789762766.946904:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 203 00000100:00100000:2.0:1789762766.946952:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8600dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014939264:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 92us (274us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762766.946961:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 204 00000100:00080000:2.0:1789762766.946997:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762766.947034:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762766.959048:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290262877616 time 290262942724 state 0x0 ref 1 00000100:00100000:0.0:1789762766.959167:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014938752:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762766.959232:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 40894464, mtime: 0x6aad9ccf, atime: 0x0, ctime: 0x6aad9ccf, blocks: 0x13807 00010000:00010000:0.0:1789762766.959245:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b03465 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.959268:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=290s v=(0 0) 00010000:00010000:0.0:1789762766.959277:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762766.959288:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b03465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762766.959315:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b03465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762766.967340:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022156928:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762766.967437:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762766.967499:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022156928 00000100:00100000:2.0:1789762766.967511:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022156928 00000100:00100000:2.0:1789762766.967588:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 87 00000100:00100000:2.0:1789762766.967604:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022156928:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762766.967671:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 339017728 pending 1703936 free 3828350976 avail 3800039424 00000100:00100000:2.0:1789762766.967782:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022156928:12345-0@lo:13:osp-pre-0-0.0 Request processed in 183us (349us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762766.967793:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 87 00000100:00100000:3.0:1789762766.967818:0:3272:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967720 gen 2 00000004:00080000:3.0:1789762766.967826:0:3272:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebad03800, transno 4294967720 00000100:00100000:3.0:1789762766.967872:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022156928:0@lo:13:osp-pre-0-0.0 00002000:00010000:3.0:1789762766.978071:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:3.0:1789762766.978078:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 79879 -> 81927 00010000:00010000:3.0:1789762766.978132:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b03465 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.978151:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b03465 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.978163:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b03465 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.978171:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762766.978176:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b03465 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.978185:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b03465 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.978203:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b03465 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762766.978210:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b03465 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762766.978288:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014938752:12345-192.168.203.48@tcp:4:dd.0 Request processed in 19199us (43637us total) trans 4294967730 rc 0/0 00000100:00100000:3.0:1789762766.978302:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 300 00000100:00100000:3.0:1789762766.978317:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290262877616 time 290262942724 state 0x2 ref 0 00000100:00100000:2.0:1789762766.981450:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762766.981493:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014939392 00000100:00100000:3.0:1789762766.981501:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014939392 00000100:00100000:3.0:1789762766.981540:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290262877616 time 290262942724 state 0x0 ref 1 00000100:00100000:3.0:1789762766.981555:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290312942724 time 290262942724 state 0x1 ref 1 00000100:00100000:0.0:1789762767.009180:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290312942724 time 290313075659 state 0x0 ref 1 00000100:00100000:0.0:1789762767.009225:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014939392:12345-192.168.203.48@tcp:4:dd.0 00000100:00080000:2.0:1789762767.009266:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789762767.009281:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 41943040, mtime: 0x6aad9ccf, atime: 0x0, ctime: 0x6aad9ccf, blocks: 0x14007 00010000:00010000:0.0:1789762767.009292:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000016873aeb/0x416193e012b0346c lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762767.009309:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=290s v=(0 0) 00010000:00010000:0.0:1789762767.009314:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762767.009320:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000016873aeb/0x416193e012b0346c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762767.009334:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000016873aeb/0x416193e012b0346c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762767.028035:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967731 is committed 00000100:00100000:2.0:1789762767.032774:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762767.032858:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014939520 00000100:00100000:0.0:1789762767.032872:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014939520 00000100:00100000:0.0:1789762767.032930:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290312942724 time 290313075659 state 0x0 ref 1 00000100:00100000:0.0:1789762767.032949:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290363075659 time 290313075659 state 0x1 ref 1 00000100:00100000:3.0:1789762767.059296:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290363075659 time 290363194034 state 0x0 ref 1 00000100:00100000:3.0:1789762767.059320:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8600d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014939520:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762767.059368:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 42991616, mtime: 0x6aad9ccf, atime: 0x0, ctime: 0x6aad9ccf, blocks: 0x14807 00010000:00010000:3.0:1789762767.059377:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e629ef2b/0x416193e012b03473 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762767.059393:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=290s v=(0 0) 00010000:00010000:3.0:1789762767.059398:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762767.059404:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e629ef2b/0x416193e012b03473 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762767.059418:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e629ef2b/0x416193e012b03473 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762767.080372:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967732 is committed 00000100:00100000:2.0:1789762767.085330:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762767.085405:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014939648 00000100:00100000:3.0:1789762767.085416:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014939648 00000100:00100000:3.0:1789762767.085495:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290363075659 time 290363194034 state 0x0 ref 1 00000100:00100000:3.0:1789762767.085531:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290413194034 time 290363194034 state 0x1 ref 1 00000001:00080000:0.0:1789762767.130251:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967733 is committed 00000100:00100000:2.0:1789762767.134492:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762767.134574:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014939776 00000100:00100000:0.0:1789762767.134599:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014939776 00000100:00100000:0.0:1789762767.134699:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290413194034 time 290413354383 state 0x0 ref 1 00000100:00100000:0.0:1789762767.134723:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290463354383 time 290413354383 state 0x1 ref 1 00000001:00080000:0.0:1789762767.181534:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967734 is committed 00000100:00100000:0.0:1789762767.186322:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762767.186418:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014939904 00000100:00100000:3.0:1789762767.186433:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014939904 00000100:00100000:3.0:1789762767.186496:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290463354383 time 290463474651 state 0x0 ref 1 00000100:00100000:3.0:1789762767.186516:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290513474651 time 290463474651 state 0x1 ref 1 00000001:00080000:2.0:1789762767.233281:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967735 is committed 00000100:00100000:0.0:1789762767.237511:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762767.281085:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967736 is committed 00000100:00100000:0.0:1789762767.285528:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762767.285644:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014940160 00000100:00100000:3.0:1789762767.285674:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014940160 00000100:00100000:3.0:1789762767.285729:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290563713419 time 290563800253 state 0x0 ref 1 00000100:00100000:3.0:1789762767.285745:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290613800253 time 290563800253 state 0x1 ref 1 00000001:00080000:2.0:1789762767.328912:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967737 is committed 00000100:00100000:0.0:1789762767.333583:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762767.379628:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967738 is committed 00000100:00100000:0.0:1789762767.383196:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762767.383262:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014940416 00000100:00100000:3.0:1789762767.383274:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014940416 00000100:00100000:3.0:1789762767.383336:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290663917869 time 290664039393 state 0x0 ref 1 00000100:00100000:3.0:1789762767.383359:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290714039393 time 290664039393 state 0x1 ref 1 00000001:00080000:2.0:1789762767.429035:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967739 is committed 00000100:00100000:2.0:1789762767.433301:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762767.477309:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967740 is committed 00000004:00080000:0.0:1789762767.479270:0:7726:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 290783180514ns, next at 291783176961ns, rc = 0 00000100:00100000:3.0:1789762767.479298:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e902df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022157056:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762767.479331:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82732a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022157312:0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.479375:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762767.479391:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762767.479433:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022157184 00000100:00100000:0.0:1789762767.479444:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022157184 00000100:00100000:2.0:1789762767.479455:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789762767.479461:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82733100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022157696:0@lo:400:kworker.0 00000100:00100000:2.0:1789762767.479475:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82732680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022157440:0@lo:400:kworker.0 00000100:00100000:0.0:1789762767.479499:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:0.0:1789762767.479514:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022157184:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.479539:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762767.479556:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762767.479630:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022157184:12345-0@lo:400:kworker.0 Request processed in 117us (261us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762767.479640:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00100000:2.0:1789762767.479651:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022157568 00000100:00100000:3.0:1789762767.479656:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022157056 00000100:00100000:2.0:1789762767.479661:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022157568 00000100:00100000:3.0:1789762767.479664:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022157056 00000100:00100000:0.0:1789762767.479697:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:0.0:1789762767.479711:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82730e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022157312:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.479742:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 88 00000100:00100000:3.0:1789762767.479762:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022157056:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762767.479791:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00002000:00000024:3.0:1789762767.479796:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 13631488 granted 97058816 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789762767.479801:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82730e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022157312:12345-0@lo:400:kworker.0 Request processed in 89us (407us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762767.479824:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:2.0:1789762767.479833:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82730a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022157440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.479871:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022157056:12345-0@lo:13:osp-pre-1-0.0 Request processed in 114us (498us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762767.479905:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 88 00000100:00100000:0.0:1789762767.479912:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82732a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022157312:0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.479946:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022157696 00000100:00100000:2.0:1789762767.479951:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82730a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022157440:12345-0@lo:400:kworker.0 Request processed in 126us (395us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762767.479953:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022157696 00000100:00100000:0.0:1789762767.479965:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82732680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022157440:0@lo:400:kworker.0 00000100:00100000:2.0:1789762767.479974:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:3.0:1789762767.480007:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 101 00000100:00100000:3.0:1789762767.480027:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022157696:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762767.480034:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762767.480084:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022157696:12345-0@lo:400:kworker.0 Request processed in 67us (546us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762767.480092:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 101 00000100:00100000:3.0:1789762767.480159:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82733100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022157696:0@lo:400:kworker.0 00000100:00100000:3.0:1789762767.480173:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e902df800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022157056:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762767.481138:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762767.481196:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014940672 00000100:00100000:3.0:1789762767.481204:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014940672 00000100:00100000:3.0:1789762767.481246:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290764195908 time 290764283959 state 0x0 ref 1 00000100:00100000:3.0:1789762767.481266:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290814283959 time 290764283959 state 0x1 ref 1 00000001:00080000:0.0:1789762767.528194:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967741 is committed 00000100:00100000:0.0:1789762767.531666:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762767.577728:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967742 is committed 00000100:00100000:2.0:1789762767.582008:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762767.582071:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014940928 00000100:00100000:3.0:1789762767.582128:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014940928 00000100:00100000:3.0:1789762767.582205:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290864435365 time 290864537050 state 0x0 ref 1 00000100:00100000:3.0:1789762767.582237:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290914537050 time 290864537050 state 0x1 ref 1 00000001:00080000:2.0:1789762767.630906:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967743 is committed 00000100:00100000:0.0:1789762767.635164:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762767.680789:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967744 is committed 00000100:00100000:0.0:1789762767.684496:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762767.684571:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014941184 00000100:00100000:3.0:1789762767.684584:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014941184 00000100:00100000:3.0:1789762767.684692:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 290964662141 time 290964752595 state 0x0 ref 1 00000100:00100000:3.0:1789762767.684711:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291014752595 time 290964752595 state 0x1 ref 1 00000001:00080000:2.0:1789762767.729669:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967745 is committed 00000100:00100000:2.0:1789762767.733731:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762767.775260:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967746 is committed 00000100:00100000:0.0:1789762767.778481:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762767.778578:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014941440 00000100:00100000:3.0:1789762767.778587:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014941440 00000100:00100000:3.0:1789762767.778645:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291064911848 time 291064972551 state 0x0 ref 1 00000100:00100000:3.0:1789762767.778659:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291114972551 time 291064972551 state 0x1 ref 1 00000001:00080000:2.0:1789762767.827171:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967747 is committed 00000100:00100000:0.0:1789762767.830979:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762767.877112:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967748 is committed 00000100:00100000:0.0:1789762767.880673:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762767.880727:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014941696 00000100:00100000:3.0:1789762767.880737:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014941696 00000100:00100000:3.0:1789762767.880795:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291165169043 time 291165231395 state 0x0 ref 1 00000100:00100000:3.0:1789762767.880806:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291215231395 time 291165231395 state 0x1 ref 1 00000001:00080000:2.0:1789762767.924880:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967749 is committed 00000100:00100000:0.0:1789762767.928295:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762767.982206:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967750 is committed 00000100:00100000:0.0:1789762767.986480:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762767.986579:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014941952 00000100:00100000:3.0:1789762767.986594:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014941952 00000100:00100000:3.0:1789762767.986682:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291265363299 time 291265482888 state 0x0 ref 1 00000100:00100000:3.0:1789762767.986700:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291315482888 time 291265482888 state 0x1 ref 1 00000001:00080000:2.0:1789762768.033631:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967751 is committed 00000100:00100000:0.0:1789762768.037770:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762768.083470:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967752 is committed 00000100:00100000:2.0:1789762768.087947:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762768.088036:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014942208 00000100:00100000:3.0:1789762768.088050:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014942208 00000100:00100000:3.0:1789762768.088141:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291365686183 time 291365787947 state 0x0 ref 1 00000100:00100000:3.0:1789762768.088162:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291415787947 time 291365787947 state 0x1 ref 1 00000001:00080000:2.0:1789762768.135252:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967753 is committed 00000100:00100000:0.0:1789762768.139868:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762768.183305:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967754 is committed 00000100:00100000:0.0:1789762768.187550:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762768.187665:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014942464 00000100:00100000:3.0:1789762768.187679:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014942464 00000100:00100000:3.0:1789762768.187752:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291465941341 time 291466031425 state 0x0 ref 1 00000100:00100000:3.0:1789762768.187772:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291516031425 time 291466031425 state 0x1 ref 1 00002000:00010000:2.0:1789762768.233709:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 67108864 -> 68157440 00002000:00010000:2.0:1789762768.233718:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 131081 -> 133129 00010000:00010000:2.0:1789762768.233733:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b03514 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762768.233750:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b03514 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762768.233784:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b03514 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762768.233794:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762768.233800:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b03514 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762768.233811:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b03514 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762768.233832:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b03514 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762768.233843:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b03514 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762768.233941:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902ddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014942464:12345-192.168.203.48@tcp:4:dd.0 Request processed in 21655us (46391us total) trans 4294967755 rc 0/0 00000100:00100000:2.0:1789762768.233956:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 325 00000100:00100000:2.0:1789762768.233970:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291516031425 time 291516158229 state 0x2 ref 0 00000100:00100000:0.0:1789762768.237822:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:3.0:1789762768.284401:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967756 is committed 00000100:00100000:0.0:1789762768.288516:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762768.288637:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014942720 00000100:00100000:3.0:1789762768.288653:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014942720 00000100:00100000:3.0:1789762768.288715:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291566158229 time 291566237640 state 0x0 ref 1 00000100:00100000:3.0:1789762768.288735:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291616237640 time 291566237640 state 0x1 ref 1 00000100:00100000:2.0:1789762768.312535:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291616237640 time 291616431051 state 0x0 ref 1 00000100:00100000:2.0:1789762768.312563:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014942720:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:2.0:1789762768.312657:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 69206016, mtime: 0x6aad9cd0, atime: 0x0, ctime: 0x6aad9cd0, blocks: 0x21009 00010000:00010000:2.0:1789762768.312670:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000faa2865c/0x416193e012b03522 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762768.312689:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=291s v=(0 0) 00010000:00010000:2.0:1789762768.312695:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762768.312703:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000faa2865c/0x416193e012b03522 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762768.312720:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000faa2865c/0x416193e012b03522 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762768.334918:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967757 is committed 00000100:00100000:2.0:1789762768.339299:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:3.0:1789762768.384579:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967758 is committed 00000100:00100000:0.0:1789762768.389185:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762768.389274:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014942976 00000100:00100000:3.0:1789762768.389289:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014942976 00000100:00100000:3.0:1789762768.389354:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291666431051 time 291666533055 state 0x0 ref 1 00000100:00100000:3.0:1789762768.389373:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291716533055 time 291666533055 state 0x1 ref 1 00000001:00080000:2.0:1789762768.435789:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967759 is committed 00000100:00100000:0.0:1789762768.440554:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762768.482077:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967760 is committed 00000100:00100000:2.0:1789762768.485504:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762768.485572:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014943232 00000100:00100000:3.0:1789762768.485584:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014943232 00000100:00100000:3.0:1789762768.485650:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291766647230 time 291766725359 state 0x0 ref 1 00000100:00100000:3.0:1789762768.485666:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291816725359 time 291766725359 state 0x1 ref 1 00000001:00080000:3.0:1789762768.528612:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967367 is committed 00000001:00080000:3.0:1789762768.528640:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967378 is committed 00000001:00080000:3.0:1789762768.528737:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967379 is committed 00000001:00080000:2.0:1789762768.535048:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967761 is committed 00000100:00100000:3.0:1789762768.539162:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762768.583757:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967762 is committed 00000100:00100000:3.0:1789762768.587945:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762768.588072:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014943488 00000100:00100000:0.0:1789762768.588121:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014943488 00000100:00100000:0.0:1789762768.588183:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291866893191 time 291867019651 state 0x0 ref 1 00000100:00100000:0.0:1789762768.588201:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291917019651 time 291867019651 state 0x1 ref 1 00000001:00080000:0.0:1789762768.634374:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967763 is committed 00000100:00100000:3.0:1789762768.638631:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762768.685681:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967764 is committed 00000100:00100000:3.0:1789762768.690253:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762768.690348:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014943744 00000100:00100000:0.0:1789762768.690364:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014943744 00000100:00100000:0.0:1789762768.690425:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 291967187926 time 291967294321 state 0x0 ref 1 00000100:00100000:0.0:1789762768.690444:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292017294321 time 291967294321 state 0x1 ref 1 00000001:00080000:0.0:1789762768.735622:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967765 is committed 00000100:00100000:0.0:1789762768.740415:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00002000:00010000:0.0:1789762768.784323:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:0.0:1789762768.784331:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 153609 -> 155657 00010000:00010000:0.0:1789762768.784350:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b03561 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.784371:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b03561 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.784389:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b03561 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.784403:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762768.784411:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b03561 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.784443:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b03561 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.784476:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b03561 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.784509:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b03561 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762768.784656:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014943872:12345-192.168.203.48@tcp:4:dd.0 Request processed in 20901us (44241us total) trans 4294967766 rc 0/0 00000100:00100000:0.0:1789762768.784667:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 336 00000100:00100000:0.0:1789762768.784680:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292067484132 time 292067626462 state 0x2 ref 0 00000100:00100000:0.0:1789762768.788706:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762768.788775:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014944000 00000100:00100000:3.0:1789762768.788788:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014944000 00000100:00100000:3.0:1789762768.788839:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292067484132 time 292067626462 state 0x0 ref 1 00000100:00100000:3.0:1789762768.788855:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292117626462 time 292067626462 state 0x1 ref 1 00010000:00010000:2.0:1789762768.823255:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762768.832475:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967767 is committed 00000100:00100000:2.0:1789762768.836047:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762768.863996:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292167784410 time 292167887989 state 0x0 ref 1 00000100:00100000:0.0:1789762768.864028:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82733100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014944128:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762768.864136:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 80740352, mtime: 0x6aad9cd0, atime: 0x0, ctime: 0x6aad9cd0, blocks: 0x26809 00010000:00010000:0.0:1789762768.864157:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b0356f lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.864186:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=292s v=(0 0) 00010000:00010000:0.0:1789762768.864192:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762768.864200:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b0356f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.864217:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b0356f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762768.879040:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967768 is committed 00000100:00100000:3.0:1789762768.881999:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762768.882078:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014944256 00000100:00100000:0.0:1789762768.882109:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014944256 00000100:00100000:0.0:1789762768.882150:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292167784410 time 292167887989 state 0x0 ref 1 00000100:00100000:0.0:1789762768.882162:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292217887989 time 292167887989 state 0x1 ref 1 00002000:00010000:0.0:1789762768.929122:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:0.0:1789762768.929128:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb blocks from disk: 159753 -> 161801 00010000:00010000:0.0:1789762768.929139:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03576 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.929149:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03576 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.929156:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03576 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.929161:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762768.929165:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03576 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.929172:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03576 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.929196:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03576 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762768.929202:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03576 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762768.929264:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014944256:12345-192.168.203.48@tcp:4:dd.0 Request processed in 15146us (47264us total) trans 4294967769 rc 0/0 00000100:00100000:0.0:1789762768.929272:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 339 00000100:00100000:0.0:1789762768.929282:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292217887989 time 292217978618 state 0x2 ref 0 00000100:00100000:3.0:1789762768.931758:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762768.979049:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967770 is committed 00000100:00100000:0.0:1789762768.982476:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762768.982540:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014944512 00000100:00100000:3.0:1789762768.982553:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014944512 00000100:00100000:3.0:1789762768.982623:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292267978618 time 292268084109 state 0x0 ref 1 00000100:00100000:3.0:1789762768.982653:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292318084109 time 292268084109 state 0x1 ref 1 00000100:00100000:0.0:1789762769.014267:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292318084109 time 292318167682 state 0x0 ref 1 00000100:00100000:0.0:1789762769.014297:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82086300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014944512:12345-192.168.203.48@tcp:4:dd.0 00000100:00080000:2.0:1789762769.014328:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789762769.014339:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 83886080, mtime: 0x6aad9cd1, atime: 0x0, ctime: 0x6aad9cd1, blocks: 0x28009 00010000:00010000:0.0:1789762769.014348:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b03584 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762769.014360:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=292s v=(0 0) 00010000:00010000:0.0:1789762769.014364:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762769.014368:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b03584 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762769.014392:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b03584 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762769.032532:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967771 is committed 00000100:00100000:3.0:1789762769.036431:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762769.083437:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967772 is committed 00000100:00100000:3.0:1789762769.087162:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762769.087240:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014944768 00000100:00100000:0.0:1789762769.087251:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014944768 00000100:00100000:0.0:1789762769.087316:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292368167682 time 292368257814 state 0x0 ref 1 00000100:00100000:0.0:1789762769.087354:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292418257814 time 292368257814 state 0x1 ref 1 00000001:00080000:0.0:1789762769.129701:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967773 is committed 00000100:00100000:2.0:1789762769.132802:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762769.177311:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967774 is committed 00000100:00100000:0.0:1789762769.180263:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762769.180312:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014945024 00000100:00100000:3.0:1789762769.180321:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014945024 00000100:00100000:3.0:1789762769.180364:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292468381995 time 292468479039 state 0x0 ref 1 00000100:00100000:3.0:1789762769.180376:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292518479039 time 292468479039 state 0x1 ref 1 00000001:00080000:0.0:1789762769.233044:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967775 is committed 00000100:00100000:0.0:1789762769.236438:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:0.0:1789762769.284482:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967776 is committed 00000100:00100000:2.0:1789762769.288059:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762769.288158:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014945280 00000100:00100000:3.0:1789762769.288173:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014945280 00000100:00100000:3.0:1789762769.288245:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292568623457 time 292568712858 state 0x0 ref 1 00000100:00100000:3.0:1789762769.288260:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292618712858 time 292568712858 state 0x1 ref 1 00000001:00080000:0.0:1789762769.336316:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967777 is committed 00000100:00100000:0.0:1789762769.340433:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762769.384498:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967778 is committed 00000100:00100000:0.0:1789762769.387771:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762769.387857:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014945536 00000100:00100000:3.0:1789762769.387870:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014945536 00000100:00100000:3.0:1789762769.387925:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292668857345 time 292668932967 state 0x0 ref 1 00000100:00100000:3.0:1789762769.387944:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292718932967 time 292668932967 state 0x1 ref 1 00000001:00080000:0.0:1789762769.436646:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967779 is committed 00000100:00100000:0.0:1789762769.441030:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:3.0:1789762769.487627:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967780 is committed 00000100:00100000:2.0:1789762769.490281:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762769.490326:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014945792 00000100:00100000:3.0:1789762769.490335:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014945792 00000100:00100000:3.0:1789762769.490374:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292769058127 time 292769123826 state 0x0 ref 1 00000100:00100000:3.0:1789762769.490385:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292819123826 time 292769123826 state 0x1 ref 1 00000001:00080000:0.0:1789762769.527235:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967781 is committed 00000100:00100000:0.0:1789762769.530003:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762769.530048:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014945920 00000100:00100000:3.0:1789762769.530056:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014945920 00000100:00100000:3.0:1789762769.530127:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292819123826 time 292819195419 state 0x0 ref 1 00000100:00100000:3.0:1789762769.530143:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292869195419 time 292819195419 state 0x1 ref 1 00000001:00080000:0.0:1789762769.576272:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967782 is committed 00000100:00100000:2.0:1789762769.578731:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762769.578771:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014946048 00000100:00100000:3.0:1789762769.578780:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014946048 00000100:00100000:3.0:1789762769.578815:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292869195419 time 292869281600 state 0x0 ref 1 00000100:00100000:3.0:1789762769.578826:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292919281600 time 292869281600 state 0x1 ref 1 00000001:00080000:0.0:1789762769.626853:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967783 is committed 00000100:00100000:0.0:1789762769.629401:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762769.629438:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014946176 00000100:00100000:3.0:1789762769.629446:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014946176 00000100:00100000:3.0:1789762769.629483:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292919281600 time 292919354355 state 0x0 ref 1 00000100:00100000:3.0:1789762769.629494:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292969354355 time 292919354355 state 0x1 ref 1 00000001:00080000:0.0:1789762769.676739:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967784 is committed 00000100:00100000:2.0:1789762769.679359:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762769.679400:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014946304 00000100:00100000:3.0:1789762769.679409:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014946304 00000100:00100000:3.0:1789762769.679447:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 292969354355 time 292969470118 state 0x0 ref 1 00000100:00100000:3.0:1789762769.679458:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293019470118 time 292969470118 state 0x1 ref 1 00000001:00080000:0.0:1789762769.727569:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967785 is committed 00000100:00100000:3.0:1789762769.730286:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762769.730366:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014946432 00000100:00100000:2.0:1789762769.730374:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014946432 00000100:00100000:2.0:1789762769.730410:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293019470118 time 293019554859 state 0x0 ref 1 00000100:00100000:2.0:1789762769.730420:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293069554859 time 293019554859 state 0x1 ref 1 00000001:00080000:3.0:1789762769.776969:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967786 is committed 00000100:00100000:3.0:1789762769.779417:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762769.779457:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014946560 00000100:00100000:0.0:1789762769.779465:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014946560 00000100:00100000:0.0:1789762769.779500:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293069554859 time 293069635981 state 0x0 ref 1 00000100:00100000:0.0:1789762769.779515:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293119635981 time 293069635981 state 0x1 ref 1 00000001:00080000:3.0:1789762769.828889:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967787 is committed 00000100:00100000:3.0:1789762769.831986:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762769.832123:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014946688 00000100:00100000:2.0:1789762769.832133:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014946688 00000100:00100000:2.0:1789762769.832192:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293119635981 time 293119721539 state 0x0 ref 1 00000100:00100000:2.0:1789762769.832209:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293169721539 time 293119721539 state 0x1 ref 1 00010000:00010000:2.0:1789762769.847204:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789762769.878459:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967788 is committed 00000100:00100000:3.0:1789762769.880801:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762769.880846:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014946816 00000100:00100000:0.0:1789762769.880855:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014946816 00000100:00100000:0.0:1789762769.880890:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293169721539 time 293169791488 state 0x0 ref 1 00000100:00100000:0.0:1789762769.880901:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293219791488 time 293169791488 state 0x1 ref 1 00000001:00080000:3.0:1789762769.927530:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967789 is committed 00000100:00100000:3.0:1789762769.929895:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762769.929968:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014946944 00000100:00100000:2.0:1789762769.929975:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014946944 00000100:00100000:2.0:1789762769.930007:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293219791488 time 293219902607 state 0x0 ref 1 00000100:00100000:2.0:1789762769.930017:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293269902607 time 293219902607 state 0x1 ref 1 00000001:00080000:3.0:1789762769.977466:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967790 is committed 00000100:00100000:3.0:1789762769.979156:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762769.979223:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014947200 00000100:00100000:0.0:1789762769.979231:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014947200 00000100:00080000:2.0:1789762769.979278:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762769.979294:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 63 00000100:00100000:0.0:1789762769.979309:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:13107:x1876702014947200:12345-192.168.203.48@tcp:35:dd.0 00000001:00080000:0.0:1789762769.979599:0:11718:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967473 00000100:00100000:0.0:1789762769.979689:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13107:x1876702014947200:12345-192.168.203.48@tcp:35:dd.0 Request processed in 384us (532us total) trans 4294967474 rc 0/0 00000100:00100000:0.0:1789762769.979699:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 63 00000020:00080000:0.0:1789762769.979705:0:11718:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789762770.025900:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762770.025981:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014947456 00000100:00100000:0.0:1789762770.025990:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014947456 00000100:00100000:0.0:1789762770.026035:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 219 00000100:00100000:0.0:1789762770.026043:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13123:x1876702014947456:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762770.026060:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762770.026105:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184c00/0x416193e012b03610 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4b1 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.026183:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03617 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.026195:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03617 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x24:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.026204:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03617 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x24:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762770.026212:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x24:0x0]: returning size 104857600 00010000:00010000:0.0:1789762770.026294:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184c00/0x416193e012b03610 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4b1 expref: 13 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.026307:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03617 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x24:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4b1 expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.026314:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x24:0x0].0x0 (ffff981ea0aa3c00) 00010000:00010000:0.0:1789762770.026316:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:0.0:1789762770.026318:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86184400, rc 0) 00000100:00100000:0.0:1789762770.026363:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839ebb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:13123:x1876702014947456:12345-192.168.203.48@tcp:101:dd.0 Request processed in 321us (470us total) trans 4294967475 rc 0/0 00000100:00100000:0.0:1789762770.026368:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 219 00000100:00100000:2.0:1789762770.027551:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762770.027623:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014947584 00000100:00100000:0.0:1789762770.027629:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014947584 00000100:00100000:0.0:1789762770.027664:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 146 00000100:00100000:0.0:1789762770.027671:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014947584:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762770.027687:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762770.027698:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000322acc95/0x416193e012b03625 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4b8 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762770.027717:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 104857600, mtime: 0x6aad9cd2, atime: 0x0, ctime: 0x6aad9cd2, blocks: 0x3200b 00010000:00010000:0.0:1789762770.027728:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=293s v=(0 0) 00010000:00010000:0.0:1789762770.027731:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762770.027738:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000322acc95/0x416193e012b03625 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4b8 expref: 6 pid: 6647 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.027748:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000322acc95/0x416193e012b03625 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4b8 expref: 6 pid: 6647 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.027755:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86185400, rc 0) 00000100:00100000:0.0:1789762770.027783:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014947584:12345-192.168.203.48@tcp:101:dd.0 Request processed in 113us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762770.027790:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 146 00000100:00100000:3.0:1789762770.029970:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762770.030020:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014947712 00000100:00100000:0.0:1789762770.030025:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014947712 00000100:00100000:0.0:1789762770.030064:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 2 deadline 287983657295 time 287883684405 state 0x0 ref 1 00000100:00100000:0.0:1789762770.030075:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 2 deadline 287983684405 time 287883684405 state 0x1 ref 1 00000100:00100000:0.0:1789762770.030110:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 2 deadline 287983684405 time 293334018735 state 0x0 ref 1 00000100:00100000:0.0:1789762770.030119:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014947712:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:0.0:1789762770.030142:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0362c lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.030151:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762770.030154:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762770.030158:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0362c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.030165:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0362c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.033779:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0362c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.033790:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0362c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.033796:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0362c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.033802:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762770.033805:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0362c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.033811:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0362c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.033821:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0362c lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762770.033832:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b0362c lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762770.033870:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014947712:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3751us (3900us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762770.033876:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 361 00000100:00100000:0.0:1789762770.033883:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 2 deadline 287983684405 time 293334018735 state 0x2 ref 0 00000100:00100000:2.0:1789762770.035825:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762770.040625:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762770.044967:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762770.140939:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 293444645468 time 293444828600 state 0x0 ref 1 00000100:00100000:2.0:1789762770.140973:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014948096:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762770.141019:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1d7e0dd/0x416193e012b03641 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762770.141038:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762770.141043:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762770.141050:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1d7e0dd/0x416193e012b03641 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762770.141065:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1d7e0dd/0x416193e012b03641 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.144609:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1d7e0dd/0x416193e012b03641 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.144620:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c1d7e0dd/0x416193e012b03641 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.144628:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c1d7e0dd/0x416193e012b03641 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.144634:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe59c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762770.144637:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1d7e0dd/0x416193e012b03641 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.144644:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1d7e0dd/0x416193e012b03641 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.144662:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c1d7e0dd/0x416193e012b03641 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.144683:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1d7e0dd/0x416193e012b03641 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762770.144760:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014948096:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3801us (99798us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762770.144768:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 364 00000100:00100000:3.0:1789762770.144776:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 293444645468 time 293444828600 state 0x2 ref 0 00000100:00100000:0.0:1789762770.147196:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762770.247619:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762770.341132:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 293644904213 time 293645021171 state 0x0 ref 1 00000100:00100000:3.0:1789762770.341171:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014948352:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762770.341232:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0364f lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.341252:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762770.341257:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762770.341263:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0364f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.341278:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0364f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.344792:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0364f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.344822:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0364f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.344830:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0364f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.344837:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762770.344841:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0364f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.344848:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0364f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.344866:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0364f lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.344871:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0364f lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762770.344927:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014948352:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3774us (97308us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762770.344935:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 366 00000100:00100000:3.0:1789762770.344944:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 293644904213 time 293645021171 state 0x2 ref 0 00000100:00100000:0.0:1789762770.347137:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762770.446655:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762770.541362:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 293845111503 time 293845246419 state 0x0 ref 1 00000100:00100000:3.0:1789762770.541422:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014948608:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762770.541482:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0365d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.541505:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762770.541512:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762770.541520:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0365d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.541539:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0365d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.545059:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0365d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.545072:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0365d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.545081:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0365d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.545120:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762770.545127:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0365d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.545140:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0365d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.545171:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0365d lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.545179:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0365d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762770.545265:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014948608:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3857us (98610us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762770.545274:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 368 00000100:00100000:3.0:1789762770.545284:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 293845111503 time 293845246419 state 0x2 ref 0 00000100:00100000:0.0:1789762770.547240:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762770.647074:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762770.741527:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 294045311064 time 294045418293 state 0x0 ref 1 00000100:00100000:3.0:1789762770.741558:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014948864:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762770.741625:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0366b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.741642:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762770.741646:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762770.741652:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0366b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.741668:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0366b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.745323:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0366b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.745339:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0366b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.745346:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0366b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.745351:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762770.745354:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0366b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.745361:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0366b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.745375:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0366b lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.745380:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b0366b lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762770.745450:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014948864:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3902us (98374us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762770.745458:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 370 00000100:00100000:3.0:1789762770.745475:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 294045311064 time 294045418293 state 0x2 ref 0 00000100:00100000:0.0:1789762770.747255:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762770.847862:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762770.871236:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762770.941731:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 294245513286 time 294245625881 state 0x0 ref 1 00000100:00100000:3.0:1789762770.941757:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014949120:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762770.941792:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03679 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.941807:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762770.941811:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762770.941815:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03679 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.941826:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03679 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.944740:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03679 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.944750:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03679 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.944756:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03679 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.944761:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762770.944773:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03679 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.944779:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03679 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.944792:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03679 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762770.944798:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03679 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762770.944844:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014949120:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3099us (96992us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762770.944851:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 372 00000100:00100000:3.0:1789762770.944858:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 294245513286 time 294245625881 state 0x2 ref 0 00000100:00100000:3.0:1789762770.946565:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762771.041955:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762771.047970:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762771.141934:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 294445710233 time 294445829526 state 0x0 ref 1 00000100:00100000:3.0:1789762771.141959:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014949376:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762771.141995:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03687 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.142009:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762771.142024:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762771.142029:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03687 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.142041:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03687 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.145083:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03687 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.145112:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03687 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.145122:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03687 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.145131:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762771.145135:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03687 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.145143:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03687 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.145155:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03687 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.145161:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03687 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762771.145212:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014949376:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3263us (97247us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762771.145219:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 374 00000100:00100000:3.0:1789762771.145226:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 294445710233 time 294445829526 state 0x2 ref 0 00000100:00100000:3.0:1789762771.146911:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762771.248841:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762771.342169:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 294645933410 time 294646064251 state 0x0 ref 1 00000100:00100000:3.0:1789762771.342193:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014949632:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762771.342228:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03695 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.342241:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762771.342245:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762771.342249:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03695 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.342260:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03695 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.345701:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03695 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.345711:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03695 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.345728:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03695 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.345733:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762771.345736:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03695 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.345743:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03695 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.345756:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03695 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.345761:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03695 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762771.345808:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014949632:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3625us (96970us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762771.345815:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 376 00000100:00100000:3.0:1789762771.345824:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 294645933410 time 294646064251 state 0x2 ref 0 00000100:00100000:2.0:1789762771.347815:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762771.449973:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762771.542447:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 294846200490 time 294846334280 state 0x0 ref 1 00000100:00100000:3.0:1789762771.542507:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014949888:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762771.542641:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036a3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.542670:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762771.542678:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762771.542689:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036a3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.542715:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036a3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.546965:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036a3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.546984:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036a3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.546995:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036a3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.547004:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762771.547009:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036a3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.547020:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036a3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.547059:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036a3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.547069:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036a3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762771.547220:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014949888:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4743us (97241us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762771.547238:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 378 00000100:00100000:3.0:1789762771.547252:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 294846200490 time 294846334280 state 0x2 ref 0 00000100:00100000:3.0:1789762771.549312:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000001:00080000:2.0:1789762771.585670:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967475 is committed 00000004:00080000:2.0:1789762771.585688:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 14 committed 00000004:00080000:2.0:1789762771.585707:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 16 committed 00000100:00100000:2.0:1789762771.585799:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022157824:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789762771.585837:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022157952:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762771.585867:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762771.585917:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762771.585984:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022157952 00000100:00080000:2.0:1789762771.585986:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762771.585995:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022157952 00000100:00080000:2.0:1789762771.586040:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762771.586060:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:0.0:1789762771.586077:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022157952:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1789762771.586131:0:6648:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:13 count 1 00002000:00010000:0.0:1789762771.586159:0:6648:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xd:0x0] initial LVB size: 8388608, mtime: 0x6aad9ccb, atime: 0x0, ctime: 0x6aad9ccb, blocks: 0x4007 00010000:00010000:0.0:1789762771.586171:0:6648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b036b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762771.586223:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=294s v=(0 0) 00010000:00010000:0.0:1789762771.586239:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762771.586260:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b036b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762771.586305:0:6648:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b036b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762771.586364:0:6648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b036b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762771.586405:0:6648:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b036b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762771.586465:0:6648:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1789762771.586492:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1789762771.586503:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b036b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762771.586551:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aac00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762771.586557:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b036b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762771.586570:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b036b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.586593:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762771.586611:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b036aa lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762771.586616:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b036b1 lrc: 1/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.586643:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762771.586645:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ccf955/0x416193e012b036b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.586654:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b036aa lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762771.586668:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762771.586672:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762771.586674:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b036aa lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762771.586675:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762771.586716:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b036aa lrc: 1/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.586724:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b036aa lrc: 0/0,0 mode: --/PW res: [0x280000400:0x9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.586742:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762771.586744:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762771.586746:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762771.587916:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022157952:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1839us (2002us total) trans 4294967791 rc 0/0 00000100:00100000:0.0:1789762771.587930:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000004:00080000:3.0:1789762771.587963:0:3272:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebad01f80/2, rc 0, transno 4294967791 00000100:00100000:3.0:1789762771.587982:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022157952:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762771.588007:0:3271:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967379 gen 2 00000004:00080000:2.0:1789762771.588014:0:3271:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981d8257fb80, transno 4294967379 00000100:00100000:2.0:1789762771.588033:0:3271:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff981d8387fb80 x1876702022157824/t4294967380(4294967380) o6->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789762787 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1789762771.588045:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981d8387fb80/2, rc 0, transno 4294967380 00000100:00100000:2.0:1789762771.588050:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022157824:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789762771.649276:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762771.742674:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 295046435018 time 295046565392 state 0x0 ref 1 00000100:00100000:3.0:1789762771.742705:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014950144:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762771.742750:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036bf lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.742769:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762771.742774:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762771.742780:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036bf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.742796:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036bf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.746465:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036bf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.746483:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036bf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.746513:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036bf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.746527:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762771.746535:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036bf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.746550:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036bf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.746595:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036bf lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.746610:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036bf lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762771.746685:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014950144:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3990us (97410us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762771.746696:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 380 00000100:00100000:3.0:1789762771.746717:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 295046435018 time 295046565392 state 0x2 ref 0 00000100:00100000:3.0:1789762771.748698:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762771.850519:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762771.895285:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762771.942951:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 295246670561 time 295246833933 state 0x0 ref 1 00000100:00100000:3.0:1789762771.943000:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfed2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014950400:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762771.943061:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036cd lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.943130:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762771.943139:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762771.943151:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036cd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.943176:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036cd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.947680:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036cd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.947698:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036cd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.947708:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036cd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.947732:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762771.947737:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036cd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.947747:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036cd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.947766:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036cd lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762771.947774:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036cd lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762771.947839:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfed2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014950400:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4859us (97322us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762771.947850:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 382 00000100:00100000:3.0:1789762771.947862:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 295246670561 time 295246833933 state 0x2 ref 0 00000100:00100000:3.0:1789762771.950279:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762772.048540:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762772.066276:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762772.066331:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762772.066399:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762772.066405:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014950784 00000100:00100000:3.0:1789762772.066413:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014950784 00000100:00100000:2.0:1789762772.066466:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762772.066496:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 288 00000100:00080000:2.0:1789762772.066506:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762772.066510:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014951040 00000100:00100000:3.0:1789762772.066534:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702014950784:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762772.066538:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014951040 00000100:00100000:2.0:1789762772.066547:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 208 00000100:00100000:2.0:1789762772.066562:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702014950912:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762772.066635:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 102 00000100:00100000:3.0:1789762772.066647:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702014950784:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 134us (373us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762772.066652:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702014950912:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 91us (317us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762772.066655:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014951040:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762772.066683:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 208 00000100:00100000:3.0:1789762772.066686:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 288 00000100:00100000:0.0:1789762772.066741:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014951040:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 75us (323us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762772.066751:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 102 00000100:00080000:2.0:1789762772.066753:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762772.066759:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762772.087237:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022158080:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762772.087294:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762772.087347:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022158080 00000100:00100000:0.0:1789762772.087356:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022158080 00000100:00100000:0.0:1789762772.087400:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 89 00000100:00100000:0.0:1789762772.087408:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022158080:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762772.087437:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439549952 pending 0 free 3746562048 avail 3744464896 00000100:00100000:0.0:1789762772.087481:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022158080:12345-0@lo:13:osp-pre-0-0.0 Request processed in 74us (187us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762772.087487:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 89 00000100:00100000:2.0:1789762772.087533:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022158080:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762772.143236:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 295446950758 time 295447129117 state 0x0 ref 1 00000100:00100000:3.0:1789762772.143285:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014950656:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762772.143329:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036db lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.143345:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762772.143349:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762772.143354:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036db lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.143366:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036db lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.147022:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036db lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.147031:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036db lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.147037:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036db lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.147042:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762772.147052:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036db lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.147059:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036db lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.147073:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036db lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.147078:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036db lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762772.147155:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014950656:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3900us (98616us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762772.147162:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 384 00000100:00100000:3.0:1789762772.147170:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 295446950758 time 295447129117 state 0x2 ref 0 00000100:00100000:3.0:1789762772.149718:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762772.250465:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762772.343490:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 295647231518 time 295647381848 state 0x0 ref 1 00000100:00100000:3.0:1789762772.343521:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86e18000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014951424:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762772.343585:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036e9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.343604:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762772.343624:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762772.343631:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036e9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.343647:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036e9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.347540:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036e9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.347552:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036e9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.347560:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036e9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.347578:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762772.347585:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036e9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.347595:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036e9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.347634:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036e9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.347646:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036e9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762772.347720:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86e18000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014951424:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4210us (97258us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762772.347730:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 386 00000100:00100000:3.0:1789762772.347740:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 295647231518 time 295647381848 state 0x2 ref 0 00000100:00100000:3.0:1789762772.350108:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762772.448887:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762772.543643:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 295847451030 time 295847538310 state 0x0 ref 1 00000100:00100000:3.0:1789762772.543665:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014951680:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762772.543702:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036f7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.543715:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762772.543719:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762772.543724:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036f7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.543735:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036f7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.546882:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036f7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.546892:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036f7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.546912:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036f7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.546918:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762772.546921:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036f7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.546928:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036f7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.546941:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036f7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.546947:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b036f7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762772.546994:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014951680:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3335us (98106us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762772.547003:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 388 00000100:00100000:3.0:1789762772.547014:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 295847451030 time 295847538310 state 0x2 ref 0 00000100:00100000:3.0:1789762772.549084:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762772.599250:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022158208:0@lo:400:kworker.0 00000100:00100000:0.0:1789762772.599281:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfedf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022158592:0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.599307:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022158464:0@lo:400:kworker.0 00000100:00100000:2.0:1789762772.599334:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762772.599363:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762772.599385:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022158208 00000100:00100000:2.0:1789762772.599406:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfede680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022158848:0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.599407:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022158208 00000100:00100000:0.0:1789762772.599424:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022158592 00000100:00100000:0.0:1789762772.599431:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022158592 00000100:00100000:2.0:1789762772.599452:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762772.599475:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 289 00000100:00100000:3.0:1789762772.599490:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022158208:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762772.599495:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762772.599496:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022158720 00000100:00100000:0.0:1789762772.599503:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022158720 00000100:00080000:2.0:1789762772.599531:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762772.599555:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:3.0:1789762772.599585:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022158208:12345-0@lo:400:kworker.0 Request processed in 80us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762772.599589:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:0.0:1789762772.599590:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022158592:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762772.599605:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86007800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022158720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.599607:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 289 00000100:00100000:0.0:1789762772.599674:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022158592:12345-0@lo:400:kworker.0 Request processed in 106us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762772.599680:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86007800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022158720:12345-0@lo:400:kworker.0 Request processed in 75us (253us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762772.599681:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762772.599692:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:2.0:1789762772.599706:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:0.0:1789762772.599717:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022158464 00000100:00100000:0.0:1789762772.599724:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022158464 00000100:00100000:3.0:1789762772.599739:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022158848 00000100:00100000:3.0:1789762772.599746:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022158848 00000100:00100000:2.0:1789762772.599758:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022158208:0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.599812:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 103 00000100:00100000:3.0:1789762772.599830:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022158848:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762772.599837:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022158464:0@lo:400:kworker.0 00000100:00100000:0.0:1789762772.599857:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfedf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022158592:0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.599891:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022158848:12345-0@lo:400:kworker.0 Request processed in 66us (439us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762772.599903:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfede680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022158848:0@lo:400:kworker.0 00000100:00100000:3.0:1789762772.599904:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 103 00000100:00100000:0.0:1789762772.649404:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762772.743819:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296047622158 time 296047715240 state 0x0 ref 1 00000100:00100000:3.0:1789762772.743846:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014951936:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762772.743882:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03705 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.743899:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762772.743906:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762772.743912:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03705 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.743928:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03705 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762772.746913:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03705 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762772.746924:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03705 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762772.746931:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03705 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762772.746936:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cb9800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762772.746940:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03705 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762772.746946:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03705 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762772.746958:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03705 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762772.746963:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c40f007f/0x416193e012b03705 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762772.747011:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014951936:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3177us (97608us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762772.747026:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 390 00000100:00100000:2.0:1789762772.747034:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296047622158 time 296047715240 state 0x2 ref 0 00000100:00100000:0.0:1789762772.749194:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762772.849376:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762772.919208:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762772.943979:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296247776528 time 296247875363 state 0x0 ref 1 00000100:00100000:2.0:1789762772.944005:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014952192:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762772.944046:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03713 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762772.944064:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762772.944069:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762772.944076:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03713 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762772.944112:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03713 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.947418:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03713 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.947430:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03713 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.947436:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03713 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.947441:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762772.947445:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03713 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.947451:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03713 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.947464:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03713 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762772.947469:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03713 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762772.947526:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014952192:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3529us (98148us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762772.947555:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 392 00000100:00100000:3.0:1789762772.947574:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296247776528 time 296247875363 state 0x2 ref 0 00000100:00100000:3.0:1789762772.949600:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762773.044155:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762773.047807:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0371a lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.047823:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0371a lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762773.047872:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014952320:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3791us (98271us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762773.047881:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 393 00000100:00100000:1.0:1789762773.047893:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296347875363 time 296347961472 state 0x2 ref 0 00000100:00100000:3.0:1789762773.049924:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762773.049964:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014952448 00000100:00100000:1.0:1789762773.049974:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014952448 00000100:00100000:1.0:1789762773.050015:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296347875363 time 296347961472 state 0x0 ref 1 00000100:00100000:1.0:1789762773.050029:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296447961472 time 296347961472 state 0x1 ref 1 00000100:00100000:3.0:1789762773.144158:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296447961472 time 296448049933 state 0x0 ref 1 00000100:00100000:3.0:1789762773.144190:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014952448:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762773.144230:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b03721 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762773.144245:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762773.144248:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762773.144253:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b03721 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762773.144263:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b03721 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762773.147356:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b03721 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762773.147367:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b03721 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762773.147395:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b03721 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762773.147401:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762773.147404:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b03721 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762773.147411:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b03721 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762773.147425:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b03721 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762773.147430:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b03721 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762773.147477:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014952448:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3300us (97555us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762773.147484:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 394 00000100:00100000:3.0:1789762773.147492:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296447961472 time 296448049933 state 0x2 ref 0 00000100:00100000:3.0:1789762773.149420:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762773.149460:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014952576 00000100:00100000:1.0:1789762773.149468:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014952576 00000100:00100000:1.0:1789762773.149510:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296447961472 time 296448049933 state 0x0 ref 1 00000100:00100000:1.0:1789762773.149521:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296548049933 time 296448049933 state 0x1 ref 1 00000100:00100000:1.0:1789762773.244238:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296548049933 time 296548132227 state 0x0 ref 1 00000100:00100000:1.0:1789762773.244267:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014952576:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762773.244308:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03728 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.244360:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762773.244366:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762773.244374:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03728 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.244391:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03728 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.248281:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03728 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.248299:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03728 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.248306:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03728 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.248313:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762773.248317:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03728 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.248324:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03728 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.248338:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03728 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.248357:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03728 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762773.248414:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014952576:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4157us (98993us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762773.248422:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 395 00000100:00100000:1.0:1789762773.248432:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296548049933 time 296548132227 state 0x2 ref 0 00000100:00100000:3.0:1789762773.250386:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762773.250429:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014952704 00000100:00100000:1.0:1789762773.250438:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014952704 00000100:00100000:1.0:1789762773.250492:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296548049933 time 296548132227 state 0x0 ref 1 00000100:00100000:1.0:1789762773.250503:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296648132227 time 296548132227 state 0x1 ref 1 00000100:00100000:3.0:1789762773.344338:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296648132227 time 296648229112 state 0x0 ref 1 00000100:00100000:3.0:1789762773.344419:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014952704:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762773.344475:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b0372f lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762773.344500:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762773.344509:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762773.344520:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b0372f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762773.344543:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b0372f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.348792:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b0372f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.348803:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b0372f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.348830:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b0372f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.348847:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762773.348852:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b0372f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.348862:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b0372f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.348882:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b0372f lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.348913:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000671f20e7/0x416193e012b0372f lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762773.349003:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014952704:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4594us (98613us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762773.349015:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 396 00000100:00100000:2.0:1789762773.349045:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296648132227 time 296648229112 state 0x2 ref 0 00000100:00100000:2.0:1789762773.351492:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762773.351540:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014952832 00000100:00100000:1.0:1789762773.351549:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014952832 00000100:00100000:1.0:1789762773.351618:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296648132227 time 296648229112 state 0x0 ref 1 00000100:00100000:1.0:1789762773.351629:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296748229112 time 296648229112 state 0x1 ref 1 00000100:00100000:1.0:1789762773.444397:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296748229112 time 296748277142 state 0x0 ref 1 00000100:00100000:1.0:1789762773.444422:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014952832:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762773.444460:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03736 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.444489:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762773.444493:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762773.444499:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03736 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.444512:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03736 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.448358:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03736 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.448368:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03736 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.448375:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03736 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.448381:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762773.448384:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03736 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.448391:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03736 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.448422:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03736 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.448440:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03736 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762773.448491:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014952832:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4076us (96998us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762773.448499:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 397 00000100:00100000:1.0:1789762773.448507:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296748229112 time 296748277142 state 0x2 ref 0 00000100:00100000:0.0:1789762773.451255:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762773.451304:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014952960 00000100:00100000:1.0:1789762773.451311:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014952960 00000100:00100000:1.0:1789762773.451345:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296748229112 time 296748277142 state 0x0 ref 1 00000100:00100000:1.0:1789762773.451356:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296848277142 time 296748277142 state 0x1 ref 1 00000100:00100000:2.0:1789762773.544505:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296848277142 time 296848401449 state 0x0 ref 1 00000100:00100000:2.0:1789762773.544531:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014952960:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762773.544587:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0373d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.544619:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762773.544623:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762773.544628:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0373d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.544640:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0373d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.547615:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0373d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.547623:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0373d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.547629:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0373d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.547634:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0a1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762773.547638:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0373d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.547645:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0373d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.547658:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0373d lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.547674:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0373d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762773.547722:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014952960:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3201us (96467us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762773.547729:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 398 00000100:00100000:2.0:1789762773.547737:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296848277142 time 296848401449 state 0x2 ref 0 00000100:00100000:3.0:1789762773.549797:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762773.549839:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014953088 00000100:00100000:1.0:1789762773.549846:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014953088 00000100:00100000:1.0:1789762773.549889:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296848277142 time 296848401449 state 0x0 ref 1 00000100:00100000:1.0:1789762773.549900:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296948401449 time 296848401449 state 0x1 ref 1 00000001:00080000:3.0:1789762773.632381:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967380 is committed 00000100:00100000:1.0:1789762773.644580:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296948401449 time 296948464810 state 0x0 ref 1 00000100:00100000:1.0:1789762773.644603:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014953088:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762773.644637:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03744 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.644661:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762773.644664:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762773.644669:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03744 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.644680:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03744 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.647355:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03744 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.647364:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03744 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.647370:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03744 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.647376:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762773.647380:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03744 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.647386:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03744 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.647399:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03744 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.647414:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03744 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762773.647459:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014953088:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2865us (97663us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762773.647466:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 399 00000100:00100000:1.0:1789762773.647473:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296948401449 time 296948464810 state 0x2 ref 0 00000100:00100000:3.0:1789762773.649418:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762773.649452:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014953216 00000100:00100000:1.0:1789762773.649460:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014953216 00000100:00100000:1.0:1789762773.649494:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 296948401449 time 296948464810 state 0x0 ref 1 00000100:00100000:1.0:1789762773.649505:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297048464810 time 296948464810 state 0x1 ref 1 00000100:00100000:2.0:1789762773.744694:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297048464810 time 297048592095 state 0x0 ref 1 00000100:00100000:2.0:1789762773.744715:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfeddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014953216:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762773.744749:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0374b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.744764:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762773.744768:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762773.744772:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0374b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.744784:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0374b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.747891:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0374b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.747935:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0374b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.747943:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0374b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.747958:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0a1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762773.747962:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0374b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.747969:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0374b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.747983:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0374b lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.747988:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0374b lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762773.748051:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfeddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014953216:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3342us (98633us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762773.748059:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 400 00000100:00100000:2.0:1789762773.748084:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297048464810 time 297048592095 state 0x2 ref 0 00000100:00100000:3.0:1789762773.750190:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762773.750271:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014953344 00000100:00100000:1.0:1789762773.750279:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014953344 00000100:00100000:1.0:1789762773.750313:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297048464810 time 297048592095 state 0x0 ref 1 00000100:00100000:1.0:1789762773.750324:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297148592095 time 297048592095 state 0x1 ref 1 00000100:00100000:1.0:1789762773.844762:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297148592095 time 297148661029 state 0x0 ref 1 00000100:00100000:1.0:1789762773.844781:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfeddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014953344:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762773.844812:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03752 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.844835:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762773.844838:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762773.844843:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03752 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.844854:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03752 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.847766:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03752 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.847776:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03752 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.847783:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03752 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.847788:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762773.847791:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03752 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.847797:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03752 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.847809:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03752 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762773.847825:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03752 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762773.847875:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfeddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014953344:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3098us (97684us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762773.847884:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 401 00000100:00100000:1.0:1789762773.847894:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297148592095 time 297148661029 state 0x2 ref 0 00000100:00100000:3.0:1789762773.849725:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762773.849761:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014953472 00000100:00100000:1.0:1789762773.849768:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014953472 00000100:00100000:1.0:1789762773.849800:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297148592095 time 297148661029 state 0x0 ref 1 00000100:00100000:1.0:1789762773.849809:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297248661029 time 297148661029 state 0x1 ref 1 00010000:00010000:2.0:1789762773.943264:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762773.944872:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297248661029 time 297248771122 state 0x0 ref 1 00000100:00100000:2.0:1789762773.944892:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014953472:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762773.944944:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03759 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.944957:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762773.944961:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762773.944965:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03759 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.944975:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03759 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.948769:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03759 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.948782:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03759 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.948790:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03759 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.948797:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0a1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762773.948801:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03759 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.948821:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03759 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.948837:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03759 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762773.948843:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03759 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762773.948898:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014953472:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4011us (99172us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762773.948907:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 402 00000100:00100000:2.0:1789762773.948916:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297248661029 time 297248771122 state 0x2 ref 0 00000100:00100000:3.0:1789762773.951143:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762773.951182:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014953600 00000100:00100000:1.0:1789762773.951190:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014953600 00000100:00100000:1.0:1789762773.951226:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297248661029 time 297248771122 state 0x0 ref 1 00000100:00100000:1.0:1789762773.951237:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297348771122 time 297248771122 state 0x1 ref 1 00000100:00100000:1.0:1789762774.045013:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297348771122 time 297348895185 state 0x0 ref 1 00000100:00100000:1.0:1789762774.045057:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014953600:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762774.045184:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03760 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.045207:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762774.045210:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762774.045214:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03760 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.045225:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03760 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.048438:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03760 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.048478:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03760 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.048490:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03760 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.048500:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762774.048506:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03760 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.048517:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03760 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.048537:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03760 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.048602:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03760 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762774.048708:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014953600:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3664us (97559us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762774.048720:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 403 00000100:00100000:1.0:1789762774.048732:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297348771122 time 297348895185 state 0x2 ref 0 00000100:00100000:3.0:1789762774.051232:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762774.051264:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014953728 00000100:00100000:1.0:1789762774.051272:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014953728 00000100:00100000:1.0:1789762774.051303:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297348771122 time 297348895185 state 0x0 ref 1 00000100:00100000:1.0:1789762774.051314:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297448895185 time 297348895185 state 0x1 ref 1 00000100:00100000:2.0:1789762774.145182:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297448895185 time 297449071044 state 0x0 ref 1 00000100:00100000:2.0:1789762774.145218:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014953728:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762774.145274:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03767 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.145304:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762774.145313:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762774.145325:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03767 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.145378:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03767 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.150689:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03767 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.150709:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03767 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.150722:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03767 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.150732:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0a1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762774.150740:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03767 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.150751:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03767 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.150777:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03767 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.150787:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03767 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762774.150885:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014953728:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5680us (99650us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762774.150899:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 404 00000100:00100000:2.0:1789762774.150918:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297448895185 time 297449071044 state 0x2 ref 0 00000100:00100000:3.0:1789762774.154465:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762774.154542:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014953856 00000100:00100000:1.0:1789762774.154578:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014953856 00000100:00100000:1.0:1789762774.154653:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297448895185 time 297449071044 state 0x0 ref 1 00000100:00100000:1.0:1789762774.154682:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297549071044 time 297449071044 state 0x1 ref 1 00000100:00100000:1.0:1789762774.245304:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297549071044 time 297549193185 state 0x0 ref 1 00000100:00100000:1.0:1789762774.245335:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfeded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014953856:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762774.245379:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0376e lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.245415:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762774.245421:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762774.245429:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0376e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.245447:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0376e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.250748:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0376e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.250768:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0376e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.250780:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0376e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.250790:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762774.250796:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0376e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.250807:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0376e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.250827:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0376e lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.250860:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0376e lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762774.250945:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfeded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014953856:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5618us (96482us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762774.250957:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 405 00000100:00100000:1.0:1789762774.250969:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297549071044 time 297549193185 state 0x2 ref 0 00000100:00100000:3.0:1789762774.254180:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762774.254247:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014953984 00000100:00100000:1.0:1789762774.254260:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014953984 00000100:00100000:1.0:1789762774.254314:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297549071044 time 297549193185 state 0x0 ref 1 00000100:00100000:1.0:1789762774.254333:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297649193185 time 297549193185 state 0x1 ref 1 00000100:00100000:2.0:1789762774.345460:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297649193185 time 297649340670 state 0x0 ref 1 00000100:00100000:2.0:1789762774.345502:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfede680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014953984:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762774.345578:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03775 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.345604:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762774.345610:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762774.345622:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03775 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.345649:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03775 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.350993:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03775 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.351022:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03775 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.351072:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03775 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.351083:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0a1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762774.351126:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03775 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.351139:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03775 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.351163:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03775 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.351173:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03775 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762774.351258:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfede680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014953984:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5769us (97079us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762774.351270:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 406 00000100:00100000:2.0:1789762774.351283:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297649193185 time 297649340670 state 0x2 ref 0 00000100:00100000:3.0:1789762774.354945:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762774.355019:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014954112 00000100:00100000:1.0:1789762774.355038:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014954112 00000100:00100000:1.0:1789762774.355148:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297649193185 time 297649340670 state 0x0 ref 1 00000100:00100000:1.0:1789762774.355168:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297749340670 time 297649340670 state 0x1 ref 1 00000100:00100000:1.0:1789762774.445531:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297749340670 time 297749419292 state 0x0 ref 1 00000100:00100000:1.0:1789762774.445572:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014954112:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762774.445624:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0377c lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.445665:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762774.445671:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762774.445681:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0377c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.445700:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0377c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.451266:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0377c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.451276:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0377c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.451283:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0377c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.451289:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762774.451293:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0377c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.451302:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0377c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.451318:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0377c lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.451335:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0377c lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762774.451396:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014954112:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5846us (96457us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762774.451403:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 407 00000100:00100000:1.0:1789762774.451410:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297749340670 time 297749419292 state 0x2 ref 0 00000100:00100000:2.0:1789762774.455217:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762774.455283:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014954240 00000100:00100000:1.0:1789762774.455299:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014954240 00000100:00100000:1.0:1789762774.455370:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297749340670 time 297749419292 state 0x0 ref 1 00000100:00100000:1.0:1789762774.455389:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297849419292 time 297749419292 state 0x1 ref 1 00000100:00100000:2.0:1789762774.545683:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297849419292 time 297849565177 state 0x0 ref 1 00000100:00100000:2.0:1789762774.545727:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014954240:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762774.545799:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03783 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.545822:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762774.545828:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762774.545835:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03783 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.545854:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03783 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.551688:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03783 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.551709:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03783 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.551722:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03783 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.551734:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0a1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762774.551740:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03783 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.551752:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03783 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.551798:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03783 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.551808:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03783 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762774.551902:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014954240:12345-192.168.203.48@tcp:3:dd.0 Request processed in 6189us (96693us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762774.551915:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 408 00000100:00100000:2.0:1789762774.551927:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297849419292 time 297849565177 state 0x2 ref 0 00000100:00100000:2.0:1789762774.555712:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762774.555792:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014954368 00000100:00100000:1.0:1789762774.555812:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014954368 00000100:00100000:1.0:1789762774.555886:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297849419292 time 297849565177 state 0x0 ref 1 00000100:00100000:1.0:1789762774.555902:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297949565177 time 297849565177 state 0x1 ref 1 00000100:00100000:1.0:1789762774.645784:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297949565177 time 297949667487 state 0x0 ref 1 00000100:00100000:1.0:1789762774.645826:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014954368:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762774.645884:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0378a lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.645921:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762774.645925:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762774.645932:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0378a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.645947:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0378a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.651294:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0378a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.651310:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0378a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.651318:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0378a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.651325:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762774.651330:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0378a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.651337:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0378a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.651355:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0378a lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.651372:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b0378a lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762774.651437:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014954368:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5629us (95732us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762774.651445:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 409 00000100:00100000:1.0:1789762774.651454:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297949565177 time 297949667487 state 0x2 ref 0 00000100:00100000:3.0:1789762774.654890:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762774.654938:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014954496 00000100:00100000:1.0:1789762774.654950:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014954496 00000100:00100000:1.0:1789762774.654997:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 297949565177 time 297949667487 state 0x0 ref 1 00000100:00100000:1.0:1789762774.655011:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298049667487 time 297949667487 state 0x1 ref 1 00000100:00100000:2.0:1789762774.745906:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298049667487 time 298049793884 state 0x0 ref 1 00000100:00100000:2.0:1789762774.745946:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014954496:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762774.746000:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03791 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.746022:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762774.746028:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762774.746036:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03791 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.746056:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03791 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.751061:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03791 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.751081:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03791 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.751154:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03791 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.751166:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0a1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762774.751175:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03791 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.751188:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03791 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.751209:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03791 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.751219:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b03791 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762774.751306:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014954496:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5374us (96413us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762774.751324:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 410 00000100:00100000:2.0:1789762774.751348:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298049667487 time 298049793884 state 0x2 ref 0 00000100:00100000:0.0:1789762774.754472:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762774.754589:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014954624 00000100:00100000:1.0:1789762774.754602:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014954624 00000100:00100000:1.0:1789762774.754668:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298049667487 time 298049793884 state 0x0 ref 1 00000100:00100000:1.0:1789762774.754684:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298149793884 time 298049793884 state 0x1 ref 1 00000100:00100000:1.0:1789762774.846001:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298149793884 time 298149890007 state 0x0 ref 1 00000100:00100000:1.0:1789762774.846036:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014954624:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762774.846120:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03798 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.846157:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762774.846163:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762774.846172:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03798 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.846192:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03798 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.849270:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03798 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.849285:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03798 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.849296:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03798 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.849306:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762774.849312:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03798 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.849324:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03798 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.849346:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03798 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762774.849363:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b03798 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762774.849421:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014954624:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3400us (94965us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762774.849430:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 411 00000100:00100000:1.0:1789762774.849438:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298149793884 time 298149890007 state 0x2 ref 0 00000100:00100000:1.0:1789762774.851355:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762774.851403:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014954752 00000100:00100000:0.0:1789762774.851413:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014954752 00000100:00100000:0.0:1789762774.851461:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298149793884 time 298149890007 state 0x0 ref 1 00000100:00100000:0.0:1789762774.851474:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298249890007 time 298149890007 state 0x1 ref 1 00000100:00100000:2.0:1789762774.946126:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298249890007 time 298250021202 state 0x0 ref 1 00000100:00100000:2.0:1789762774.946153:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014954752:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762774.946188:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0379f lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.946202:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762774.946206:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762774.946210:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0379f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.946222:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0379f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.949652:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0379f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.949663:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0379f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.949670:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0379f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.949676:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0a1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762774.949694:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0379f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.949701:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0379f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.949716:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0379f lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762774.949721:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b87aa6f3/0x416193e012b0379f lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762774.949770:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014954752:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3628us (98416us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762774.949778:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 412 00000100:00100000:2.0:1789762774.949786:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298249890007 time 298250021202 state 0x2 ref 0 00000100:00100000:3.0:1789762774.951757:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762774.951803:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014954880 00000100:00100000:1.0:1789762774.951813:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014954880 00000100:00100000:1.0:1789762774.951872:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298249890007 time 298250021202 state 0x0 ref 1 00000100:00100000:1.0:1789762774.951890:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298350021202 time 298250021202 state 0x1 ref 1 00010000:00010000:3.0:1789762774.967212:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762775.046178:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298350021202 time 298350068581 state 0x0 ref 1 00000100:00100000:1.0:1789762775.046231:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81381880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014954880:12345-192.168.203.48@tcp:3:dd.0 00000100:00080000:0.0:1789762775.046236:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762775.046282:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037a6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.046304:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.046311:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.046320:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037a6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.046348:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037a6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.049792:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037a6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.049803:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037a6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.049810:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037a6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.049817:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762775.049821:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037a6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.049829:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037a6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.049848:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037a6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.049854:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037a6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762775.049922:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81381880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014954880:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3724us (98170us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762775.049932:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 413 00000100:00100000:1.0:1789762775.049942:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298350021202 time 298350068581 state 0x2 ref 0 00000100:00100000:0.0:1789762775.052171:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762775.052280:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014955008 00000100:00100000:1.0:1789762775.052291:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014955008 00000100:00100000:1.0:1789762775.052347:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298350021202 time 298350068581 state 0x0 ref 1 00000100:00100000:1.0:1789762775.052364:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298450068581 time 298350068581 state 0x1 ref 1 00000100:00100000:1.0:1789762775.146218:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298450068581 time 298450112762 state 0x0 ref 1 00000100:00100000:1.0:1789762775.146244:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014955008:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762775.146300:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037ad lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.146319:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.146323:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.146328:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037ad lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.146342:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037ad lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.155247:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037ad lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.155259:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037ad lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.155268:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037ad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.155275:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762775.155279:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037ad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.155286:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037ad lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.155304:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037ad lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.155311:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037ad lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762775.155364:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014955008:12345-192.168.203.48@tcp:3:dd.0 Request processed in 9128us (103194us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762775.155371:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 414 00000100:00100000:1.0:1789762775.155386:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298450068581 time 298450112762 state 0x2 ref 0 00000100:00100000:3.0:1789762775.157974:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762775.158017:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014955136 00000100:00100000:1.0:1789762775.158028:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014955136 00000100:00100000:1.0:1789762775.158083:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298450068581 time 298450112762 state 0x0 ref 1 00000100:00100000:1.0:1789762775.158128:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298550112762 time 298450112762 state 0x1 ref 1 00000001:00080000:0.0:1789762775.170716:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967791 is committed 00000100:00100000:1.0:1789762775.246317:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298550112762 time 298550201452 state 0x0 ref 1 00000100:00100000:1.0:1789762775.246360:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81382680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014955136:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762775.246419:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037b4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.246440:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.246445:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.246450:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037b4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.246466:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037b4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.250651:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037b4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.250660:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037b4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.250675:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037b4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.250681:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762775.250685:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037b4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.250692:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037b4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.250711:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037b4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.250718:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037b4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762775.250776:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81382680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014955136:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4435us (92804us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762775.250784:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 415 00000100:00100000:1.0:1789762775.250792:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298550112762 time 298550201452 state 0x2 ref 0 00000100:00100000:3.0:1789762775.254228:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762775.254282:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014955264 00000100:00100000:0.0:1789762775.254293:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014955264 00000100:00100000:0.0:1789762775.254348:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298550112762 time 298550201452 state 0x0 ref 1 00000100:00100000:0.0:1789762775.254362:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298650201452 time 298550201452 state 0x1 ref 1 00000100:00100000:1.0:1789762775.346447:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298650201452 time 298650327584 state 0x0 ref 1 00000100:00100000:1.0:1789762775.346482:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81382d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014955264:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762775.346586:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037bb lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.346606:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.346611:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.346617:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.346632:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.351731:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.351748:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037bb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.351759:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037bb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.351769:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762775.351776:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037bb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.351789:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037bb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.351829:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037bb lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.351839:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037bb lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762775.351924:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81382d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014955264:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5452us (97691us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762775.351936:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 416 00000100:00100000:1.0:1789762775.351949:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298650201452 time 298650327584 state 0x2 ref 0 00000100:00100000:0.0:1789762775.354361:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762775.354399:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014955392 00000100:00100000:1.0:1789762775.354409:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014955392 00000100:00100000:1.0:1789762775.354459:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298650201452 time 298650327584 state 0x0 ref 1 00000100:00100000:1.0:1789762775.354471:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298750327584 time 298650327584 state 0x1 ref 1 00000100:00100000:1.0:1789762775.446562:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298750327584 time 298750435999 state 0x0 ref 1 00000100:00100000:1.0:1789762775.446599:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014955392:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762775.446657:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.446677:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.446702:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.446708:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.446724:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.451727:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.451742:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.451754:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.451764:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762775.451771:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.451782:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.451801:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.451808:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762775.451890:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014955392:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5306us (97530us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762775.451900:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 417 00000100:00100000:1.0:1789762775.451910:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298750327584 time 298750435999 state 0x2 ref 0 00000100:00100000:1.0:1789762775.455104:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762775.455188:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014955520 00000100:00100000:0.0:1789762775.455200:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014955520 00000100:00100000:0.0:1789762775.455249:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298750327584 time 298750435999 state 0x0 ref 1 00000100:00100000:0.0:1789762775.455262:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298850435999 time 298750435999 state 0x1 ref 1 00000100:00100000:1.0:1789762775.546749:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298850435999 time 298850632472 state 0x0 ref 1 00000100:00100000:1.0:1789762775.546786:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014955520:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762775.546861:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.546885:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.546892:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.546901:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.546920:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762775.550943:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762775.550960:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762775.550967:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762775.550973:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762775.550978:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762775.550988:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762775.551009:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762775.551019:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b037c9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762775.551125:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014955520:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4350us (96037us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762775.551134:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 418 00000100:00100000:3.0:1789762775.551145:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298850435999 time 298850632472 state 0x2 ref 0 00000100:00100000:2.0:1789762775.553808:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762775.553918:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014955648 00000100:00100000:0.0:1789762775.553953:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014955648 00000100:00100000:0.0:1789762775.554022:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298850435999 time 298850632472 state 0x0 ref 1 00000100:00100000:0.0:1789762775.554040:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298950632472 time 298850632472 state 0x1 ref 1 00000100:00100000:1.0:1789762775.646914:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298950632472 time 298950787663 state 0x0 ref 1 00000100:00100000:1.0:1789762775.646973:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014955648:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762775.647059:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037d0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.647090:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.647144:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.647158:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037d0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.647181:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037d0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.650989:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037d0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.651001:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037d0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.651008:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037d0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.651014:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762775.651020:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037d0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.651027:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037d0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.651061:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037d0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.651070:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037d0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762775.651178:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014955648:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4223us (97370us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762775.651186:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 419 00000100:00100000:1.0:1789762775.651195:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298950632472 time 298950787663 state 0x2 ref 0 00000100:00100000:0.0:1789762775.654229:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762775.654339:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014955776 00000100:00100000:3.0:1789762775.654352:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014955776 00000100:00100000:3.0:1789762775.654406:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 298950632472 time 298950787663 state 0x0 ref 1 00000100:00100000:3.0:1789762775.654429:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299050787663 time 298950787663 state 0x1 ref 1 00000100:00100000:3.0:1789762775.746958:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299050787663 time 299050854707 state 0x0 ref 1 00000100:00100000:3.0:1789762775.746990:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014955776:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762775.747027:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b037d7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762775.747042:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762775.747046:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762775.747050:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b037d7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762775.747064:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b037d7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.750517:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b037d7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.750559:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b037d7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.750572:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b037d7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.750581:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c6f200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762775.750587:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b037d7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.750600:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b037d7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.750664:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b037d7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.750678:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b037d7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762775.750805:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014955776:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3827us (96579us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762775.750820:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 420 00000100:00100000:1.0:1789762775.750834:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299050787663 time 299050854707 state 0x2 ref 0 00000100:00100000:0.0:1789762775.753642:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762775.753706:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014955904 00000100:00100000:1.0:1789762775.753719:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014955904 00000100:00100000:1.0:1789762775.753780:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299050787663 time 299050854707 state 0x0 ref 1 00000100:00100000:1.0:1789762775.753798:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299150854707 time 299050854707 state 0x1 ref 1 00000100:00100000:1.0:1789762775.847146:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299150854707 time 299150993118 state 0x0 ref 1 00000100:00100000:1.0:1789762775.847196:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014955904:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762775.847265:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037de lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.847293:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.847319:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.847330:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037de lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.847353:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037de lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.851347:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037de lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.851365:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037de lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.851381:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037de lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.851393:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762775.851419:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037de lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.851437:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037de lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.851470:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037de lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.851485:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037de lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762775.851641:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014955904:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4465us (98002us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762775.851656:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 421 00000100:00100000:1.0:1789762775.851670:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299150854707 time 299150993118 state 0x2 ref 0 00000100:00100000:0.0:1789762775.853871:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762775.853922:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014956032 00000100:00100000:3.0:1789762775.853935:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014956032 00000100:00100000:3.0:1789762775.853994:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299150854707 time 299150993118 state 0x0 ref 1 00000100:00100000:3.0:1789762775.854007:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299250993118 time 299150993118 state 0x1 ref 1 00000100:00100000:1.0:1789762775.947254:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299250993118 time 299251131718 state 0x0 ref 1 00000100:00100000:1.0:1789762775.947306:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014956032:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762775.947377:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037e5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.947402:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.947408:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762775.947417:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037e5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.947437:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037e5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.951156:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037e5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.951172:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037e5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.951207:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037e5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.951215:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762775.951219:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037e5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.951228:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037e5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.951248:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037e5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762775.951255:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037e5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762775.951332:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014956032:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4051us (97461us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762775.951342:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 422 00000100:00100000:1.0:1789762775.951353:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299250993118 time 299251131718 state 0x2 ref 0 00000100:00100000:3.0:1789762775.954082:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762775.954167:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014956160 00000100:00100000:1.0:1789762775.954182:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014956160 00000100:00100000:1.0:1789762775.954241:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299250993118 time 299251131718 state 0x0 ref 1 00000100:00100000:1.0:1789762775.954272:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299351131718 time 299251131718 state 0x1 ref 1 00010000:00010000:2.0:1789762775.991330:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762776.047379:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299351131718 time 299351259590 state 0x0 ref 1 00000100:00100000:1.0:1789762776.047435:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81383800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014956160:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762776.047511:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037ec lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.047537:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.047562:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.047579:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.047604:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.051597:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.051615:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037ec lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.051626:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.051663:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762776.051669:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.051681:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037ec lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.051703:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037ec lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.051711:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037ec lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762776.051801:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81383800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014956160:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4390us (97722us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762776.051813:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 423 00000100:00100000:1.0:1789762776.051826:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299351131718 time 299351259590 state 0x2 ref 0 00000100:00100000:3.0:1789762776.055635:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762776.055713:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014956288 00000100:00100000:0.0:1789762776.055723:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014956288 00000100:00100000:0.0:1789762776.055794:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299351131718 time 299351259590 state 0x0 ref 1 00000100:00100000:0.0:1789762776.055812:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299451259590 time 299351259590 state 0x1 ref 1 00000100:00100000:1.0:1789762776.147515:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299451259590 time 299451403751 state 0x0 ref 1 00000100:00100000:1.0:1789762776.147566:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014956288:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762776.147631:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037f3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.147677:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.147682:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.147688:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037f3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.147704:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037f3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.151250:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037f3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.151262:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037f3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.151270:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.151276:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762776.151280:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.151287:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037f3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.151303:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037f3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.151322:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037f3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762776.151388:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014956288:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3853us (95759us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762776.151396:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 424 00000100:00100000:1.0:1789762776.151404:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299451259590 time 299451403751 state 0x2 ref 0 00000100:00100000:3.0:1789762776.153909:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762776.153950:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014956416 00000100:00100000:1.0:1789762776.153962:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014956416 00000100:00100000:1.0:1789762776.154008:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299451259590 time 299451403751 state 0x0 ref 1 00000100:00100000:1.0:1789762776.154022:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299551403751 time 299451403751 state 0x1 ref 1 00000100:00100000:1.0:1789762776.247679:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299551403751 time 299551558409 state 0x0 ref 1 00000100:00100000:1.0:1789762776.247722:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014956416:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762776.247780:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037fa lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.247804:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.247809:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.247816:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037fa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.247865:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037fa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.251977:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037fa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.251993:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037fa lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.252006:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037fa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.252014:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762776.252019:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037fa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.252029:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037fa lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.252051:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037fa lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.252058:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b037fa lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762776.252189:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014956416:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4485us (98284us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762776.252202:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 425 00000100:00100000:1.0:1789762776.252214:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299551403751 time 299551558409 state 0x2 ref 0 00000100:00100000:0.0:1789762776.255791:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762776.255843:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014956544 00000100:00100000:3.0:1789762776.255854:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014956544 00000100:00100000:3.0:1789762776.255906:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299551403751 time 299551558409 state 0x0 ref 1 00000100:00100000:3.0:1789762776.255919:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299651558409 time 299551558409 state 0x1 ref 1 00000100:00100000:1.0:1789762776.347805:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299651558409 time 299651683490 state 0x0 ref 1 00000100:00100000:1.0:1789762776.347858:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014956544:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762776.347924:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03801 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.347947:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.347952:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.347960:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03801 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.347978:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03801 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.351371:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03801 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.351386:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03801 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.351421:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03801 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.351430:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762776.351435:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03801 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.351444:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03801 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.351464:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03801 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.351473:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03801 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762776.351573:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014956544:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3738us (95786us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762776.351586:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 426 00000100:00100000:1.0:1789762776.351597:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299651558409 time 299651683490 state 0x2 ref 0 00000100:00100000:0.0:1789762776.354205:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762776.354258:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014956672 00000100:00100000:1.0:1789762776.354271:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014956672 00000100:00100000:1.0:1789762776.354327:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299651558409 time 299651683490 state 0x0 ref 1 00000100:00100000:1.0:1789762776.354356:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299751683490 time 299651683490 state 0x1 ref 1 00000100:00100000:1.0:1789762776.447921:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299751683490 time 299751792123 state 0x0 ref 1 00000100:00100000:1.0:1789762776.447980:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014956672:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762776.448050:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03808 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.448078:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.448119:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.448128:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03808 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.448150:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03808 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.452565:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03808 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.452594:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03808 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.452615:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03808 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.452668:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762776.452675:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03808 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.452689:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03808 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.452714:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03808 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.452725:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03808 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762776.452823:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014956672:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4866us (98619us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762776.452839:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 427 00000100:00100000:1.0:1789762776.452853:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299751683490 time 299751792123 state 0x2 ref 0 00000100:00100000:0.0:1789762776.456232:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762776.456259:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014956800 00000100:00100000:3.0:1789762776.456269:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014956800 00000100:00100000:3.0:1789762776.456329:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299751683490 time 299751792123 state 0x0 ref 1 00000100:00100000:3.0:1789762776.456348:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299851792123 time 299751792123 state 0x1 ref 1 00000100:00100000:1.0:1789762776.548053:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299851792123 time 299851931819 state 0x0 ref 1 00000100:00100000:1.0:1789762776.548158:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014956800:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762776.548238:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0380f lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.548307:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.548313:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.548320:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0380f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.548338:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0380f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.552270:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0380f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.552287:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0380f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.552298:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0380f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.552306:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762776.552311:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0380f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.552320:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0380f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.552341:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0380f lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.552366:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0380f lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762776.552452:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014956800:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4370us (96235us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762776.552462:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 428 00000100:00100000:1.0:1789762776.552473:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299851792123 time 299851931819 state 0x2 ref 0 00000100:00100000:3.0:1789762776.554660:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762776.554723:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014956928 00000100:00100000:1.0:1789762776.554736:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014956928 00000100:00100000:1.0:1789762776.554791:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299851792123 time 299851931819 state 0x0 ref 1 00000100:00100000:1.0:1789762776.554808:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299951931819 time 299851931819 state 0x1 ref 1 00000100:00100000:1.0:1789762776.648197:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299951931819 time 299952075440 state 0x0 ref 1 00000100:00100000:1.0:1789762776.648248:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead6e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014956928:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762776.648322:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03816 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.648350:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.648358:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.648368:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03816 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.648421:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03816 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.652823:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03816 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.652845:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03816 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.652861:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03816 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.652876:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762776.652884:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03816 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.652900:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03816 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.652929:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03816 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.652943:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03816 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762776.653074:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead6e6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014956928:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4846us (98418us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762776.653147:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 429 00000100:00100000:1.0:1789762776.653166:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299951931819 time 299952075440 state 0x2 ref 0 00000100:00100000:2.0:1789762776.655275:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762776.655335:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014957056 00000100:00100000:3.0:1789762776.655348:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014957056 00000100:00100000:3.0:1789762776.655401:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 299951931819 time 299952075440 state 0x0 ref 1 00000100:00100000:3.0:1789762776.655414:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300052075440 time 299952075440 state 0x1 ref 1 00000100:00100000:1.0:1789762776.748291:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300052075440 time 300052182088 state 0x0 ref 1 00000100:00100000:1.0:1789762776.748325:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014957056:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762776.748378:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0381d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.748397:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762776.748401:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.748407:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0381d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.748424:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0381d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.751768:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0381d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.751780:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0381d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.751812:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0381d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.751819:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762776.751824:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0381d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.751835:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0381d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.751858:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0381d lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.751869:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0381d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762776.751953:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014957056:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3643us (96682us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762776.751962:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 430 00000100:00100000:1.0:1789762776.751971:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300052075440 time 300052182088 state 0x2 ref 0 00000100:00100000:0.0:1789762776.753934:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762776.753983:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014957184 00000100:00100000:1.0:1789762776.753993:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014957184 00000100:00100000:1.0:1789762776.754044:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300052075440 time 300052182088 state 0x0 ref 1 00000100:00100000:1.0:1789762776.754074:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300152182088 time 300052182088 state 0x1 ref 1 00000100:00100000:1.0:1789762776.848433:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300152182088 time 300152313348 state 0x0 ref 1 00000100:00100000:1.0:1789762776.848495:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014957184:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762776.848570:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03824 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.848592:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.848598:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.848622:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03824 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.848642:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03824 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.852729:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03824 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.852743:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03824 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.852753:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03824 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.852774:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762776.852780:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03824 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.852806:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03824 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.852826:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03824 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.852833:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03824 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762776.852901:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014957184:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4442us (98966us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762776.852910:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 431 00000100:00100000:1.0:1789762776.852918:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300152182088 time 300152313348 state 0x2 ref 0 00000100:00100000:3.0:1789762776.855203:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762776.855253:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014957312 00000100:00100000:0.0:1789762776.855269:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014957312 00000100:00100000:0.0:1789762776.855317:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300152182088 time 300152313348 state 0x0 ref 1 00000100:00100000:0.0:1789762776.855329:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300252313348 time 300152313348 state 0x1 ref 1 00000100:00100000:1.0:1789762776.948598:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300252313348 time 300252466634 state 0x0 ref 1 00000100:00100000:1.0:1789762776.948654:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf512a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014957312:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762776.948711:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0382b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.948755:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.948777:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762776.948783:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0382b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.948797:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0382b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.952346:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0382b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.952374:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0382b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.952383:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0382b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.952391:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762776.952395:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0382b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.952405:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0382b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.952423:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0382b lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762776.952470:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0382b lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762776.952615:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf512a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014957312:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3992us (97416us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762776.952627:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 432 00000100:00100000:1.0:1789762776.952639:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300252313348 time 300252466634 state 0x2 ref 0 00000100:00100000:3.0:1789762776.955034:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762776.955083:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014957440 00000100:00100000:1.0:1789762776.955114:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014957440 00000100:00100000:1.0:1789762776.955164:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300252313348 time 300252466634 state 0x0 ref 1 00000100:00100000:1.0:1789762776.955180:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300352466634 time 300252466634 state 0x1 ref 1 00010000:00010000:0.0:1789762777.015215:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762777.048687:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300352466634 time 300352573945 state 0x0 ref 1 00000100:00100000:1.0:1789762777.048744:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014957440:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762777.048801:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03832 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789762777.048818:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762777.048824:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.048833:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.048843:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03832 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.048896:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03832 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.053013:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03832 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.053024:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03832 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.053031:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03832 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.053038:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762777.053043:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03832 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.053051:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03832 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.053070:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03832 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.053078:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03832 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762777.053213:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014957440:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4504us (98182us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762777.053243:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 433 00000100:00100000:1.0:1789762777.053260:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300352466634 time 300352573945 state 0x2 ref 0 00000100:00100000:0.0:1789762777.055472:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762777.055558:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014957568 00000100:00100000:3.0:1789762777.055569:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014957568 00000100:00100000:3.0:1789762777.055650:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300352466634 time 300352573945 state 0x0 ref 1 00000100:00100000:3.0:1789762777.055679:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300452573945 time 300352573945 state 0x1 ref 1 00000100:00100000:1.0:1789762777.148946:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300452573945 time 300452829586 state 0x0 ref 1 00000100:00100000:1.0:1789762777.148986:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014957568:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762777.149048:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03839 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.149071:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.149078:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.149086:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03839 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.149103:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03839 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.153010:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03839 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.153026:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03839 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.153063:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03839 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.153072:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762777.153078:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03839 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.153115:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03839 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.153138:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03839 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.153147:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03839 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762777.153225:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014957568:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4256us (97768us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762777.153236:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 434 00000100:00100000:1.0:1789762777.153248:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300452573945 time 300452829586 state 0x2 ref 0 00000100:00100000:2.0:1789762777.155420:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762777.155489:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014957696 00000100:00100000:1.0:1789762777.155504:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014957696 00000100:00100000:1.0:1789762777.155616:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300452573945 time 300452829586 state 0x0 ref 1 00000100:00100000:1.0:1789762777.155661:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300552829586 time 300452829586 state 0x1 ref 1 00000100:00100000:0.0:1789762777.186620:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762777.186702:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762777.186748:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762777.186750:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014957824 00000100:00100000:1.0:1789762777.186766:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014957824 00000100:00100000:2.0:1789762777.186787:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014957952 00000100:00100000:2.0:1789762777.186802:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014957952 00000100:00100000:3.0:1789762777.186826:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014958208 00000100:00100000:3.0:1789762777.186835:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014958208 00000100:00100000:0.0:1789762777.186868:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762777.186914:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 290 00000100:00100000:2.0:1789762777.186924:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 213 00000100:00100000:1.0:1789762777.186944:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702014957824:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762777.186946:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 214 00000100:00100000:2.0:1789762777.186948:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702014957952:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:0.0:1789762777.186950:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762777.186984:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014958208:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762777.187036:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014958080 00000100:00100000:0.0:1789762777.187063:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014958080 00000100:00100000:1.0:1789762777.187092:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702014957824:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 158us (486us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762777.187121:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702014957952:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 150us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762777.187125:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014958208:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 114us (341us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762777.187145:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 214 00000100:00100000:1.0:1789762777.187174:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 290 00000100:00100000:2.0:1789762777.187175:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 213 00000100:00100000:0.0:1789762777.187206:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 104 00000100:00100000:0.0:1789762777.187229:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014958080:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:1.0:1789762777.187276:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762777.187294:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014958080:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 73us (434us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762777.187309:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 104 00000100:00100000:1.0:1789762777.207303:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86005500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022158976:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762777.207387:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762777.207464:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022158976 00000100:00100000:0.0:1789762777.207474:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022158976 00000100:00080000:2.0:1789762777.207550:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762777.207579:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 91 00000100:00100000:0.0:1789762777.207599:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86006300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022158976:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762777.207651:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439549952 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762777.207792:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86006300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022158976:12345-0@lo:13:osp-pre-0-0.0 Request processed in 190us (399us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762777.207798:0:3272:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967791 gen 2 00000004:00080000:2.0:1789762777.207808:0:3272:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebad01f80, transno 4294967791 00000100:00100000:0.0:1789762777.207812:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 91 00000100:00100000:2.0:1789762777.207846:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86005500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022158976:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762777.249065:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300552829586 time 300552947321 state 0x0 ref 1 00000100:00100000:1.0:1789762777.249162:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014957696:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762777.249221:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03840 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.249245:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.249252:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.249260:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03840 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.249279:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03840 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.253340:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03840 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.253368:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03840 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.253385:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03840 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.253401:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762777.253411:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03840 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.253428:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03840 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.253460:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03840 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.253470:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03840 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762777.253588:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014957696:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4440us (98167us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762777.253608:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 435 00000100:00100000:1.0:1789762777.253640:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300552829586 time 300552947321 state 0x2 ref 0 00000100:00100000:3.0:1789762777.256742:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762777.256826:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014958336 00000100:00100000:0.0:1789762777.256843:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014958336 00000100:00100000:0.0:1789762777.256918:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300552829586 time 300552947321 state 0x0 ref 1 00000100:00100000:0.0:1789762777.256939:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300652947321 time 300552947321 state 0x1 ref 1 00000100:00100000:1.0:1789762777.349167:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300652947321 time 300653053392 state 0x0 ref 1 00000100:00100000:1.0:1789762777.349211:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014958336:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762777.349266:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03847 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.349289:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.349294:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.349302:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03847 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.349318:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03847 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.353005:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03847 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.353017:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03847 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.353026:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03847 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.353035:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762777.353064:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03847 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.353079:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03847 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.353129:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03847 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.353136:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03847 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762777.353204:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014958336:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4007us (96473us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762777.353217:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 436 00000100:00100000:1.0:1789762777.353230:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300652947321 time 300653053392 state 0x2 ref 0 00000100:00100000:3.0:1789762777.355318:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762777.355373:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014958464 00000100:00100000:1.0:1789762777.355383:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014958464 00000100:00100000:1.0:1789762777.355434:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300652947321 time 300653053392 state 0x0 ref 1 00000100:00100000:1.0:1789762777.355447:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300753053392 time 300653053392 state 0x1 ref 1 00000100:00100000:1.0:1789762777.449236:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300753053392 time 300753132010 state 0x0 ref 1 00000100:00100000:1.0:1789762777.449259:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014958464:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762777.449311:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0384e lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.449325:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.449328:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.449333:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0384e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.449344:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0384e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.452459:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0384e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.452469:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0384e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.452476:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0384e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.452481:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762777.452484:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0384e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.452490:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0384e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.452513:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0384e lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.452520:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0384e lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762777.452578:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014958464:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3325us (97265us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762777.452585:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 437 00000100:00100000:1.0:1789762777.452592:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300753053392 time 300753132010 state 0x2 ref 0 00000100:00100000:0.0:1789762777.455123:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762777.455174:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014958592 00000100:00100000:3.0:1789762777.455183:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014958592 00000100:00100000:3.0:1789762777.455222:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300753053392 time 300753132010 state 0x0 ref 1 00000100:00100000:3.0:1789762777.455233:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300853132010 time 300753132010 state 0x1 ref 1 00000100:00100000:1.0:1789762777.549371:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300853132010 time 300853266391 state 0x0 ref 1 00000100:00100000:1.0:1789762777.549396:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014958592:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762777.549432:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03855 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.549446:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.549449:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.549454:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03855 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.549465:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03855 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.552830:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03855 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.552839:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03855 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.552845:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03855 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.552851:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762777.552855:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03855 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.552861:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03855 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.552875:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03855 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.552880:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03855 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762777.552930:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014958592:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3545us (97844us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762777.552938:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 438 00000100:00100000:1.0:1789762777.552952:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300853132010 time 300853266391 state 0x2 ref 0 00000100:00100000:2.0:1789762777.554880:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762777.554918:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014958720 00000100:00100000:1.0:1789762777.554926:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014958720 00000100:00100000:1.0:1789762777.554958:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300853132010 time 300853266391 state 0x0 ref 1 00000100:00100000:1.0:1789762777.554968:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300953266391 time 300853266391 state 0x1 ref 1 00000100:00100000:1.0:1789762777.649454:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300953266391 time 300953350315 state 0x0 ref 1 00000100:00100000:1.0:1789762777.649478:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014958720:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762777.649524:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0385c lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.649552:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.649555:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.649560:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0385c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.649572:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0385c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.652871:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0385c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.652882:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0385c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.652901:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0385c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.652906:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762777.652909:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0385c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.652916:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0385c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.652928:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0385c lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.652934:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0385c lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762777.652981:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014958720:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3512us (98101us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762777.652995:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 439 00000100:00100000:1.0:1789762777.653005:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300953266391 time 300953350315 state 0x2 ref 0 00000100:00100000:3.0:1789762777.655349:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762777.655404:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014958848 00000100:00100000:0.0:1789762777.655417:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014958848 00000100:00100000:0.0:1789762777.655514:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 300953266391 time 300953350315 state 0x0 ref 1 00000100:00100000:0.0:1789762777.655599:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301053350315 time 300953350315 state 0x1 ref 1 00000100:00100000:1.0:1789762777.719262:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022159104:0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.719267:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022159488:0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.719268:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc437b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022159232:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762777.719302:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebf51e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022159360:0@lo:400:kworker.0 00000100:00100000:1.0:1789762777.719356:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762777.719369:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762777.719370:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762777.719404:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022159104 00000100:00100000:3.0:1789762777.719412:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022159104 00000100:00100000:1.0:1789762777.719436:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022159616:0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.719438:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022159744:0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.719469:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 291 00000100:00100000:2.0:1789762777.719479:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762777.719492:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022159104:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.719499:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022159488 00000100:00100000:0.0:1789762777.719506:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022159488 00000100:00100000:1.0:1789762777.719513:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762777.719543:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022159232 00000100:00100000:2.0:1789762777.719550:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022159232 00000100:00100000:3.0:1789762777.719587:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022159104:12345-0@lo:400:kworker.0 Request processed in 105us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762777.719592:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022159104:0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.719596:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022159744 00000100:00100000:0.0:1789762777.719607:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022159744 00000100:00100000:3.0:1789762777.719613:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 291 00000100:00100000:2.0:1789762777.719617:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 92 00000100:00100000:2.0:1789762777.719638:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022159232:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762777.719643:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022159616 00000100:00100000:1.0:1789762777.719649:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022159616 00000100:00100000:0.0:1789762777.719660:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:0.0:1789762777.719674:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022159488:12345-0@lo:400:kworker.0 00002000:00000024:2.0:1789762777.719674:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 13631488 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762777.719694:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:1.0:1789762777.719695:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 105 00000100:00100000:3.0:1789762777.719713:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022159744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762777.719717:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022159616:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.719741:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022159232:12345-0@lo:13:osp-pre-1-0.0 Request processed in 115us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762777.719749:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 92 00000100:00100000:0.0:1789762777.719777:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022159488:12345-0@lo:400:kworker.0 Request processed in 106us (419us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762777.719781:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762777.719785:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00080000:2.0:1789762777.719786:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762777.719804:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022159744:12345-0@lo:400:kworker.0 Request processed in 96us (326us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762777.719816:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:1.0:1789762777.719817:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86007480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022159616:12345-0@lo:400:kworker.0 Request processed in 105us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762777.719827:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 105 00000100:00100000:2.0:1789762777.719837:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdf8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022159744:0@lo:400:kworker.0 00000100:00100000:0.0:1789762777.719849:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022159488:0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.719861:0:3274:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967380 gen 2 00000100:00100000:3.0:1789762777.719865:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:2.0:1789762777.719866:0:3274:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981d8387fb80, transno 4294967380 00000100:00100000:1.0:1789762777.719887:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022159616:0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.719899:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc437b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022159232:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762777.719905:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022159872:0@lo:400:kworker.0 00000100:00100000:1.0:1789762777.719930:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022159360 00000100:00100000:1.0:1789762777.719935:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022159360 00000100:00100000:3.0:1789762777.719945:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762777.719972:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:1.0:1789762777.719984:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022159360:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.720000:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022159872 00000100:00100000:3.0:1789762777.720004:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022159872 00000100:00100000:1.0:1789762777.720039:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022159360:12345-0@lo:400:kworker.0 Request processed in 57us (176us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762777.720040:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 106 00000100:00100000:1.0:1789762777.720054:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:3.0:1789762777.720054:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022159872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.720082:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebf51e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022159360:0@lo:400:kworker.0 00000100:00100000:2.0:1789762777.720150:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdf9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022159872:0@lo:400:kworker.0 00000100:00100000:3.0:1789762777.720153:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022159872:12345-0@lo:400:kworker.0 Request processed in 102us (205us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762777.720161:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 106 00000100:00100000:1.0:1789762777.749588:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301053350315 time 301053476162 state 0x0 ref 1 00000100:00100000:1.0:1789762777.749606:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014958848:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762777.749637:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03863 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.749661:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762777.749665:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.749679:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03863 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.749692:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03863 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.753106:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03863 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.753122:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03863 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.753134:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03863 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.753144:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762777.753149:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03863 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.753157:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03863 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.753172:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03863 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.753178:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03863 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762777.753239:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014958848:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3637us (97891us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762777.753248:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 440 00000100:00100000:1.0:1789762777.753255:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301053350315 time 301053476162 state 0x2 ref 0 00000100:00100000:3.0:1789762777.755281:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762777.755315:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014958976 00000100:00100000:1.0:1789762777.755321:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014958976 00000100:00100000:1.0:1789762777.755364:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301053350315 time 301053476162 state 0x0 ref 1 00000100:00100000:1.0:1789762777.755375:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301153476162 time 301053476162 state 0x1 ref 1 00000100:00100000:1.0:1789762777.849683:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301153476162 time 301153578595 state 0x0 ref 1 00000100:00100000:1.0:1789762777.849710:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014958976:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762777.849747:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0386a lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.849762:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.849765:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.849770:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0386a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.849781:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0386a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.852513:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0386a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.852538:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0386a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.852547:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0386a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.852553:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762777.852556:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0386a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.852563:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0386a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.852576:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0386a lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.852581:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0386a lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762777.852638:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014958976:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2939us (97358us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762777.852645:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 441 00000100:00100000:1.0:1789762777.852660:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301153476162 time 301153578595 state 0x2 ref 0 00000100:00100000:2.0:1789762777.854705:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762777.854756:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014959104 00000100:00100000:0.0:1789762777.854769:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014959104 00000100:00100000:0.0:1789762777.854828:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301153476162 time 301153578595 state 0x0 ref 1 00000100:00100000:0.0:1789762777.854841:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301253578595 time 301153578595 state 0x1 ref 1 00000100:00100000:1.0:1789762777.949794:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301253578595 time 301253689863 state 0x0 ref 1 00000100:00100000:1.0:1789762777.949821:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014959104:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762777.949857:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03871 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.949871:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.949874:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762777.949879:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03871 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.949890:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03871 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.952914:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03871 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.952925:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03871 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.952932:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03871 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.952938:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762777.952948:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03871 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.952956:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03871 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.952971:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03871 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762777.952977:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03871 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762777.953034:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014959104:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3222us (98328us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762777.953045:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 442 00000100:00100000:1.0:1789762777.953054:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301253578595 time 301253689863 state 0x2 ref 0 00000100:00100000:0.0:1789762777.955001:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762777.955047:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014959232 00000100:00100000:1.0:1789762777.955055:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014959232 00000100:00100000:1.0:1789762777.955118:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301253578595 time 301253689863 state 0x0 ref 1 00000100:00100000:1.0:1789762777.955135:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301353689863 time 301253689863 state 0x1 ref 1 00010000:00010000:2.0:1789762778.039222:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762778.049853:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301353689863 time 301353751487 state 0x0 ref 1 00000100:00100000:1.0:1789762778.049875:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014959232:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762778.049923:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03878 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.049939:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.049942:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.049946:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03878 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.049957:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03878 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.053236:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03878 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.053247:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03878 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.053254:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03878 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.053259:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762778.053263:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03878 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.053269:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03878 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.053293:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03878 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.053299:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03878 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762778.053350:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014959232:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3482us (98355us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762778.053356:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 443 00000100:00100000:1.0:1789762778.053364:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301353689863 time 301353751487 state 0x2 ref 0 00000100:00100000:3.0:1789762778.055294:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762778.055351:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014959360 00000100:00100000:0.0:1789762778.055366:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014959360 00000100:00100000:0.0:1789762778.055424:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301353689863 time 301353751487 state 0x0 ref 1 00000100:00100000:0.0:1789762778.055453:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301453751487 time 301353751487 state 0x1 ref 1 00000100:00100000:1.0:1789762778.149982:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301453751487 time 301453876043 state 0x0 ref 1 00000100:00100000:1.0:1789762778.150012:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014959360:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762778.150056:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0387f lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.150071:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.150075:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.150079:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0387f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.150123:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0387f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.153430:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0387f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.153446:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0387f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.153457:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0387f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.153464:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762778.153468:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0387f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.153475:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0387f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.153488:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0387f lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.153494:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0387f lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762778.153560:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014959360:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3560us (98266us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762778.153567:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 444 00000100:00100000:1.0:1789762778.153582:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301453751487 time 301453876043 state 0x2 ref 0 00000100:00100000:3.0:1789762778.155444:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762778.155480:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014959488 00000100:00100000:1.0:1789762778.155487:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014959488 00000100:00100000:1.0:1789762778.155520:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301453751487 time 301453876043 state 0x0 ref 1 00000100:00100000:1.0:1789762778.155545:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301553876043 time 301453876043 state 0x1 ref 1 00000100:00100000:1.0:1789762778.250068:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301553876043 time 301553964398 state 0x0 ref 1 00000100:00100000:1.0:1789762778.250116:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014959488:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762778.250154:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03886 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.250171:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.250176:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.250184:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03886 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.250200:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03886 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.253605:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03886 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.253619:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03886 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.253638:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03886 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.253649:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762778.253655:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03886 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.253665:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03886 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.253679:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03886 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.253685:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03886 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762778.253737:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014959488:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3653us (98293us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762778.253743:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 445 00000100:00100000:1.0:1789762778.253750:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301553876043 time 301553964398 state 0x2 ref 0 00000100:00100000:3.0:1789762778.255821:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762778.255875:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014959616 00000100:00100000:0.0:1789762778.255889:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014959616 00000100:00100000:0.0:1789762778.255947:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301553876043 time 301553964398 state 0x0 ref 1 00000100:00100000:0.0:1789762778.255961:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301653964398 time 301553964398 state 0x1 ref 1 00000100:00100000:1.0:1789762778.350219:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301653964398 time 301654110402 state 0x0 ref 1 00000100:00100000:1.0:1789762778.350247:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014959616:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762778.350304:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0388d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.350330:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.350336:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.350341:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0388d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.350353:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0388d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.353668:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0388d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.353683:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0388d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.353695:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0388d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.353705:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762778.353709:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0388d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.353719:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0388d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.353751:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0388d lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.353760:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0388d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762778.353822:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702014959616:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3585us (98001us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762778.353833:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 446 00000100:00100000:1.0:1789762778.353846:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301653964398 time 301654110402 state 0x2 ref 0 00000100:00100000:3.0:1789762778.355862:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762778.355908:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014959744 00000100:00100000:1.0:1789762778.355918:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014959744 00000100:00100000:1.0:1789762778.355968:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301653964398 time 301654110402 state 0x0 ref 1 00000100:00100000:1.0:1789762778.355983:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301754110402 time 301654110402 state 0x1 ref 1 00000100:00100000:1.0:1789762778.450313:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301754110402 time 301754208265 state 0x0 ref 1 00000100:00100000:1.0:1789762778.450339:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014959744:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762778.450376:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03894 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.450389:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.450405:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.450411:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03894 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.450422:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03894 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.453682:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03894 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.453693:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03894 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.453699:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03894 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.453704:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762778.453708:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03894 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.453714:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03894 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.453726:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03894 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.453731:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b03894 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762778.453790:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014959744:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3461us (97929us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762778.453798:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 447 00000100:00100000:1.0:1789762778.453805:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301754110402 time 301754208265 state 0x2 ref 0 00000100:00100000:3.0:1789762778.455799:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762778.455847:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014959872 00000100:00100000:0.0:1789762778.455855:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014959872 00000100:00100000:0.0:1789762778.455892:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301754110402 time 301754208265 state 0x0 ref 1 00000100:00100000:0.0:1789762778.455903:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301854208265 time 301754208265 state 0x1 ref 1 00000100:00100000:1.0:1789762778.550446:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301854208265 time 301854342078 state 0x0 ref 1 00000100:00100000:1.0:1789762778.550472:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014959872:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762778.550519:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0389b lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.550551:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.550556:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.550563:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0389b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.550576:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0389b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.553992:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0389b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.554007:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0389b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.554040:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0389b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.554049:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762778.554055:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0389b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.554066:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0389b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.554108:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0389b lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.554119:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b0389b lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762778.554183:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014959872:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3722us (98390us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762778.554193:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 448 00000100:00100000:1.0:1789762778.554203:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301854208265 time 301854342078 state 0x2 ref 0 00000100:00100000:2.0:1789762778.556506:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762778.556567:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014960000 00000100:00100000:1.0:1789762778.556575:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014960000 00000100:00100000:1.0:1789762778.556614:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301854208265 time 301854342078 state 0x0 ref 1 00000100:00100000:1.0:1789762778.556638:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301954342078 time 301854342078 state 0x1 ref 1 00000100:00100000:1.0:1789762778.650618:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301954342078 time 301954512542 state 0x0 ref 1 00000100:00100000:1.0:1789762778.650645:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014960000:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762778.650682:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.650696:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.650699:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.650703:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.650714:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.653653:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.653664:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.653670:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.653691:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762778.653695:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.653701:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.653713:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.653719:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762778.653769:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014960000:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3134us (97264us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762778.653778:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 449 00000100:00100000:1.0:1789762778.653786:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301954342078 time 301954512542 state 0x2 ref 0 00000100:00100000:0.0:1789762778.655698:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762778.655743:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014960128 00000100:00100000:3.0:1789762778.655757:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014960128 00000100:00100000:3.0:1789762778.655813:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 301954342078 time 301954512542 state 0x0 ref 1 00000100:00100000:3.0:1789762778.655842:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302054512542 time 301954512542 state 0x1 ref 1 00000100:00100000:1.0:1789762778.750694:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302054512542 time 302054591990 state 0x0 ref 1 00000100:00100000:1.0:1789762778.750716:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014960128:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762778.750750:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.750789:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762778.750794:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.750801:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.750818:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.754162:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.754173:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.754180:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.754186:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762778.754189:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.754196:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.754209:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.754225:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038a9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762778.754273:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014960128:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3564us (98576us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762778.754280:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 450 00000100:00100000:1.0:1789762778.754287:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302054512542 time 302054591990 state 0x2 ref 0 00000100:00100000:0.0:1789762778.756152:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762778.756191:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014960256 00000100:00100000:1.0:1789762778.756198:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014960256 00000100:00100000:1.0:1789762778.756234:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302054512542 time 302054591990 state 0x0 ref 1 00000100:00100000:1.0:1789762778.756244:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302154591990 time 302054591990 state 0x1 ref 1 00000100:00100000:1.0:1789762778.850776:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302154591990 time 302154666649 state 0x0 ref 1 00000100:00100000:1.0:1789762778.850809:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014960256:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762778.850854:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.850873:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.850878:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.850885:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.850921:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.854440:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.854454:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.854466:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.854476:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762778.854481:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.854492:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.854512:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.854535:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762778.854614:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014960256:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3817us (98462us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762778.854626:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 451 00000100:00100000:1.0:1789762778.854637:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302154591990 time 302154666649 state 0x2 ref 0 00000100:00100000:2.0:1789762778.856714:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762778.856770:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014960384 00000100:00100000:3.0:1789762778.856787:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014960384 00000100:00100000:3.0:1789762778.856856:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302154591990 time 302154666649 state 0x0 ref 1 00000100:00100000:3.0:1789762778.856875:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302254666649 time 302154666649 state 0x1 ref 1 00000100:00100000:1.0:1789762778.950924:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302254666649 time 302254808579 state 0x0 ref 1 00000100:00100000:1.0:1789762778.950950:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014960384:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762778.950986:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.951011:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.951015:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762778.951020:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.951031:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.955712:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.955723:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.955747:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.955754:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762778.955758:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.955765:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.955779:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762778.955784:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038b7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762778.955836:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014960384:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4897us (99125us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762778.955843:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 452 00000100:00100000:1.0:1789762778.955850:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302254666649 time 302254808579 state 0x2 ref 0 00000100:00100000:2.0:1789762778.958722:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762778.958771:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014960512 00000100:00100000:1.0:1789762778.958780:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014960512 00000100:00100000:1.0:1789762778.958843:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302254666649 time 302254808579 state 0x0 ref 1 00000100:00100000:1.0:1789762778.958871:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302354808579 time 302254808579 state 0x1 ref 1 00000100:00100000:1.0:1789762779.050998:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302354808579 time 302354894315 state 0x0 ref 1 00000100:00100000:1.0:1789762779.051039:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf51e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014960512:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762779.051077:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038be lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762779.051123:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762779.051125:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762779.051130:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762779.051135:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038be lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.051147:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038be lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.054587:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038be lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.054597:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038be lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.054603:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038be lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.054624:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762779.054628:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038be lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.054635:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038be lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.054647:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038be lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.054652:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038be lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762779.054704:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf51e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014960512:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3689us (95991us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762779.054711:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 453 00000100:00100000:1.0:1789762779.054718:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302354808579 time 302354894315 state 0x2 ref 0 00000100:00100000:0.0:1789762779.056992:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762779.057055:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014960640 00000100:00100000:3.0:1789762779.057064:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014960640 00000100:00100000:3.0:1789762779.057118:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302354808579 time 302354894315 state 0x0 ref 1 00000100:00100000:3.0:1789762779.057129:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302454894315 time 302354894315 state 0x1 ref 1 00010000:00010000:3.0:1789762779.063249:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762779.151142:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302454894315 time 302454996899 state 0x0 ref 1 00000100:00100000:1.0:1789762779.151179:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014960640:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762779.151217:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038c5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.151249:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762779.151253:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762779.151257:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038c5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.151268:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038c5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.155258:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038c5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.155277:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038c5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.155294:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038c5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.155303:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762779.155311:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038c5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.155322:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038c5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.155347:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038c5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.155371:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001f99d421/0x416193e012b038c5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762779.155464:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014960640:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4303us (98468us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762779.155488:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 454 00000100:00100000:2.0:1789762779.155511:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302454894315 time 302454996899 state 0x2 ref 0 00000100:00100000:0.0:1789762779.158467:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762779.158543:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014960768 00000100:00100000:1.0:1789762779.158554:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014960768 00000100:00100000:1.0:1789762779.158603:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302454894315 time 302454996899 state 0x0 ref 1 00000100:00100000:1.0:1789762779.158620:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302554996899 time 302454996899 state 0x1 ref 1 00000100:00100000:1.0:1789762779.251210:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302554996899 time 302555103040 state 0x0 ref 1 00000100:00100000:1.0:1789762779.251241:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014960768:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762779.251284:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038cc lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.251302:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762779.251307:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762779.251313:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038cc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.251348:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038cc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.254578:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038cc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.254592:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038cc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.254603:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038cc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.254612:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762779.254617:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038cc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.254628:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038cc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.254647:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038cc lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.254657:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038cc lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762779.254743:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014960768:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3513us (96279us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762779.254754:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 455 00000100:00100000:1.0:1789762779.254773:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302554996899 time 302555103040 state 0x2 ref 0 00000100:00100000:0.0:1789762779.257168:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762779.257231:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014960896 00000100:00100000:3.0:1789762779.257238:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014960896 00000100:00100000:3.0:1789762779.257272:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302554996899 time 302555103040 state 0x0 ref 1 00000100:00100000:3.0:1789762779.257283:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302655103040 time 302555103040 state 0x1 ref 1 00000100:00100000:2.0:1789762779.351301:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302655103040 time 302655196444 state 0x0 ref 1 00000100:00100000:2.0:1789762779.351325:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014960896:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762779.351360:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038d3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.351375:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762779.351379:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762779.351384:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038d3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.351395:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038d3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.354778:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038d3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.354805:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038d3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.354812:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038d3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.354818:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762779.354822:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038d3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.354829:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038d3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.354842:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038d3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.354848:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038d3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762779.354898:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014960896:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3581us (97731us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762779.354912:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 456 00000100:00100000:2.0:1789762779.354919:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302655103040 time 302655196444 state 0x2 ref 0 00000100:00100000:3.0:1789762779.357344:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762779.357393:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014961024 00000100:00100000:1.0:1789762779.357401:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014961024 00000100:00100000:1.0:1789762779.357437:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302655103040 time 302655196444 state 0x0 ref 1 00000100:00100000:1.0:1789762779.357448:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302755196444 time 302655196444 state 0x1 ref 1 00000100:00100000:1.0:1789762779.451353:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302755196444 time 302755249584 state 0x0 ref 1 00000100:00100000:1.0:1789762779.451373:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014961024:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762779.451408:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038da lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.451422:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762779.451426:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762779.451430:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038da lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.451441:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038da lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.454759:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038da lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.454769:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038da lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.454786:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038da lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.454791:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762779.454794:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038da lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.454800:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038da lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.454813:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038da lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.454817:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038da lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762779.454866:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014961024:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3498us (97522us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762779.454877:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 457 00000100:00100000:1.0:1789762779.454896:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302755196444 time 302755249584 state 0x2 ref 0 00000100:00100000:0.0:1789762779.457211:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762779.457254:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014961152 00000100:00100000:3.0:1789762779.457262:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014961152 00000100:00100000:3.0:1789762779.457302:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302755196444 time 302755249584 state 0x0 ref 1 00000100:00100000:3.0:1789762779.457323:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302855249584 time 302755249584 state 0x1 ref 1 00000100:00100000:2.0:1789762779.551458:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302855249584 time 302855353779 state 0x0 ref 1 00000100:00100000:2.0:1789762779.551493:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014961152:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762779.551545:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038e1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.551569:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762779.551573:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762779.551583:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038e1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.551595:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038e1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.555004:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038e1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.555015:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038e1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.555022:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038e1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.555030:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762779.555036:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038e1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.555061:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038e1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.555080:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038e1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.555106:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038e1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762779.555154:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014961152:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3667us (97946us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762779.555161:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 458 00000100:00100000:2.0:1789762779.555170:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302855249584 time 302855353779 state 0x2 ref 0 00000100:00100000:0.0:1789762779.557469:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762779.557536:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014961280 00000100:00100000:1.0:1789762779.557545:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014961280 00000100:00100000:1.0:1789762779.557586:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302855249584 time 302855353779 state 0x0 ref 1 00000100:00100000:1.0:1789762779.557597:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302955353779 time 302855353779 state 0x1 ref 1 00000100:00100000:1.0:1789762779.651533:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302955353779 time 302955416562 state 0x0 ref 1 00000100:00100000:1.0:1789762779.651574:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebf47dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014961280:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762779.651610:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038e8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.651624:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762779.651628:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762779.651633:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038e8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.651643:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038e8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.655257:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038e8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.655267:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038e8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.655281:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.655287:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762779.655291:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.655297:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038e8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.655322:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038e8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.655328:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b038e8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762779.655378:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebf47dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014961280:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3829us (97909us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762779.655386:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 459 00000100:00100000:1.0:1789762779.655393:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302955353779 time 302955416562 state 0x2 ref 0 00000100:00100000:0.0:1789762779.657941:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762779.657987:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014961408 00000100:00100000:3.0:1789762779.657997:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014961408 00000100:00100000:3.0:1789762779.658032:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 302955353779 time 302955416562 state 0x0 ref 1 00000100:00100000:3.0:1789762779.658043:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 303055416562 time 302955416562 state 0x1 ref 1 00000100:00100000:2.0:1789762779.751631:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 303055416562 time 303055526737 state 0x0 ref 1 00000100:00100000:2.0:1789762779.751654:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014961408:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762779.751689:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038ef lrc: 3/1,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.751702:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762779.751706:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762779.751711:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038ef lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.751739:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038ef lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.755314:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038ef lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.755326:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038ef lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.755333:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038ef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.755339:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762779.755343:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038ef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.755349:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038ef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.755363:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038ef lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.755369:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b038ef lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762779.755429:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014961408:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3783us (97488us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762779.755436:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 460 00000100:00100000:2.0:1789762779.755444:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 303055416562 time 303055526737 state 0x2 ref 0 00000100:00100000:3.0:1789762779.782318:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762779.782389:0:5792:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014961536 00000100:00100000:0.0:1789762779.782399:0:5792:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014961536 00000100:00100000:0.0:1789762779.782452:0:5792:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 16 00000100:00100000:0.0:1789762779.782460:0:5792:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:5593:x1876702014961536:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762779.782484:0:5792:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762779.782534:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000322acc95/0x416193e012b03625 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4b8 expref: 6 pid: 6647 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.782560:0:5792:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762779.782607:0:5792:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:5593:x1876702014961536:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 148us (296us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762779.782613:0:5792:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 16 00000100:00100000:3.0:1789762779.791724:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762779.791767:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014961664 00000100:00100000:0.0:1789762779.791774:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014961664 00000100:00080000:2.0:1789762779.791823:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762779.791833:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 220 00000100:00100000:0.0:1789762779.791845:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:12244:x1876702014961664:12345-192.168.203.48@tcp:101:bash.0 00010000:00010000:0.0:1789762779.791862:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762779.791881:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b038f6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4bf expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.791940:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b038fd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.791955:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b038fd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.791968:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b038fd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762779.792002:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00000004:00200000:0.0:1789762779.792006:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762779.792028:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b038fd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.792065:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b038f6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4bf expref: 11 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.792082:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b038fd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4bf expref: 10 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.792125:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981e83a46000) 00010000:00010000:0.0:1789762779.792128:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762779.792133:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aba00, rc 0) 00000100:00100000:0.0:1789762779.792169:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:12244:x1876702014961664:12345-192.168.203.48@tcp:101:bash.0 Request processed in 326us (446us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762779.792177:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 220 00000100:00100000:1.0:1789762779.796253:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762779.796315:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014961920 00000100:00100000:2.0:1789762779.796322:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014961920 00000100:00100000:2.0:1789762779.796359:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 221 00000100:00100000:2.0:1789762779.796367:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13136:x1876702014961920:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:2.0:1789762779.796376:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762779.796387:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b03904 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4c6 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762779.796446:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00010000:00010000:2.0:1789762779.796651:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b03904 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4c6 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.796662:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:2.0:1789762779.796665:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762779.796670:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030600/0x416193e012b03904 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4c6 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.796684:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e80030600, rc 0) 00000100:00100000:2.0:1789762779.796717:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13136:x1876702014961920:12345-192.168.203.48@tcp:101:rm.0 Request processed in 351us (465us total) trans 4294967476 rc 301/301 00000100:00100000:2.0:1789762779.796731:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 221 00000100:00100000:3.0:1789762779.798323:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762779.798362:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014962048 00000100:00100000:2.0:1789762779.798368:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014962048 00000100:00100000:2.0:1789762779.798401:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 64 00000100:00100000:2.0:1789762779.798411:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b19050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13136:x1876702014962048:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:2.0:1789762779.800484:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b19050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13136:x1876702014962048:12345-192.168.203.48@tcp:37:rm.0 Request processed in 2074us (2160us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762779.800493:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 64 00000100:00100000:1.0:1789762779.802005:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762779.802051:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014962304 00000100:00100000:0.0:1789762779.802058:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014962304 00000100:00100000:0.0:1789762779.802121:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 65 00000100:00100000:0.0:1789762779.802132:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13136:x1876702014962304:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:0.0:1789762779.802261:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:13136:x1876702014962304:12345-192.168.203.48@tcp:35:rm.0 Request processed in 130us (254us total) trans 4294967477 rc 0/0 00000100:00100000:0.0:1789762779.802271:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 65 00000100:00100000:3.0:1789762779.803134:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762779.803212:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014962560 00000100:00100000:1.0:1789762779.803217:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014962560 00000100:00100000:1.0:1789762779.803254:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 222 00000100:00100000:1.0:1789762779.803261:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:13136:x1876702014962560:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:1.0:1789762779.803271:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762779.803280:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03912 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4cd expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.803316:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b03919 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.803324:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b03919 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.803331:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b03919 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762779.803337:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00010000:00010000:1.0:1789762779.803382:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03912 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4cd expref: 15 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.803393:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b03919 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4cd expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.803398:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981e83a46000) 00010000:00010000:1.0:1789762779.803400:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1789762779.803402:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb5a00, rc 0) 00000100:00100000:1.0:1789762779.803432:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13136:x1876702014962560:12345-192.168.203.48@tcp:101:rm.0 Request processed in 171us (300us total) trans 4294967478 rc 0/0 00000100:00100000:1.0:1789762779.803439:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 222 00000100:00100000:3.0:1789762779.804333:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762779.804387:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014962688 00000100:00100000:0.0:1789762779.804395:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014962688 00000100:00100000:0.0:1789762779.804433:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 223 00000100:00100000:0.0:1789762779.804443:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13136:x1876702014962688:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:0.0:1789762779.804480:0:6611:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7596933120 00000100:00100000:0.0:1789762779.804539:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13136:x1876702014962688:12345-192.168.203.48@tcp:41:rm.0 Request processed in 97us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762779.804556:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 223 00000100:00100000:0.0:1789762779.805246:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762779.805283:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014962944 00000100:00100000:2.0:1789762779.805288:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014962944 00000100:00100000:2.0:1789762779.805318:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 66 00000100:00100000:2.0:1789762779.805329:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13136:x1876702014962944:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:2.0:1789762779.805506:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:13136:x1876702014962944:12345-192.168.203.48@tcp:35:rm.0 Request processed in 178us (259us total) trans 4294967479 rc 0/0 00000100:00100000:2.0:1789762779.805529:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 66 00000100:00100000:3.0:1789762779.806140:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762779.806160:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014963200 00000100:00100000:2.0:1789762779.806166:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014963200 00000100:00100000:2.0:1789762779.806197:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 224 00000100:00100000:2.0:1789762779.806206:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:13136:x1876702014963200:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:2.0:1789762779.806232:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762779.806247:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184400/0x416193e012b03617 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x24:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4b1 expref: 15 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.806265:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185e00/0x416193e012b03338 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4aa expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.806288:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x24:0x0].0x0 (ffff981ea0aa3c00) 00010000:00010000:2.0:1789762779.806303:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b038fd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4bf expref: 13 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.806312:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981e83a46000) 00010000:00010000:2.0:1789762779.806315:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762779.806326:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031e00/0x416193e012b03927 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.806335:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031e00/0x416193e012b03927 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.806342:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031e00/0x416193e012b03927 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.806356:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b0392e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.806364:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b0392e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.806371:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b0392e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.806667:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03935 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x24:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.806676:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03935 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x24:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.806693:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03935 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x24:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789762779.807596:0:5804:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].14, 64 off9088 00000004:00080000:2.0:1789762779.807607:0:5804:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 303111519104ns, next at 304111516407ns, rc = 0 00010000:00010000:2.0:1789762779.807649:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0393c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807659:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0393c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x24:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807668:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0393c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x24:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807675:0:5804:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0393c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x24:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807682:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0393c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x24:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807688:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0393c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x24:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807706:0:5804:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762779.807717:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03935 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x24:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807723:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03935 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x24:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807730:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03935 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x24:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807744:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762779.807747:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762779.807749:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03935 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x24:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.807757:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0393c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x24:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807760:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03935 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x24:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.807776:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762779.807784:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0393c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x24:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807784:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x24:0x0].0x0 (ffff981e82a26f00) 00010000:00010000:2.0:1789762779.807792:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762779.807800:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03935 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x24:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.807801:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0393c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x24:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807820:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030400/0x416193e012b03935 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x24:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.807828:0:6663:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x24:0x0].0x0 (ffff981e82a26f00) 00010000:00010000:0.0:1789762779.807833:0:6663:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762779.807838:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0393c lrc: 1/0,0 mode: --/PW res: [0x200000401:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807839:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031e00/0x416193e012b03927 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.807856:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e80030a00/0x416193e012b0393c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807858:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031e00/0x416193e012b03927 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807877:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031e00/0x416193e012b03927 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807884:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762779.807887:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031e00/0x416193e012b03927 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.807889:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762779.807894:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762779.807898:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762779.807899:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031e00/0x416193e012b03927 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807919:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981e83a46000) 00010000:00010000:2.0:1789762779.807922:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762779.807925:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031e00/0x416193e012b03927 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762779.807931:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031e00/0x416193e012b03927 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762779.807942:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005f236189 reply state 00000000cbecd70e transno 4294967480 ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b0392e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762779.807950:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762779.807991:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13136:x1876702014963200:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1786us (1853us total) trans 4294967480 rc 0/0 00000100:00100000:2.0:1789762779.807999:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 224 00010000:00010000:0.0:1789762779.808626:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b0392e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.808633:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b0392e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.808638:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b0392e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.808647:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762779.808650:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b0392e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.808654:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b0392e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.808661:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4009f790 (ffff981e82a26c00) 00010000:00010000:0.0:1789762779.808663:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762779.808665:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b0392e lrc: 2/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762779.808669:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e80031000/0x416193e012b0392e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762779.809109:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762779.809158:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014963456 00000100:00100000:0.0:1789762779.809164:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014963456 00000100:00100000:0.0:1789762779.809197:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 67 00000100:00100000:0.0:1789762779.809206:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:13136:x1876702014963456:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:0.0:1789762779.809308:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13136:x1876702014963456:12345-192.168.203.48@tcp:35:rm.0 Request processed in 101us (222us total) trans 4294967481 rc 0/0 00000100:00100000:0.0:1789762779.809323:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 67 00000100:00100000:3.0:1789762779.809980:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762779.810011:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014963712 00000100:00100000:1.0:1789762779.810023:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014963712 00000100:00100000:1.0:1789762779.810057:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 225 00000100:00100000:1.0:1789762779.810067:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13136:x1876702014963712:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:1.0:1789762779.810116:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1789762779.810129:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b03919 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4cd expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810142:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0331c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4a3 expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810150:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981e83a46000) 00010000:00010000:1.0:1789762779.810158:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b032ba lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd48e expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810165:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26800) 00010000:00010000:1.0:1789762779.810169:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762779.810179:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b03943 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810185:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b03943 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810190:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b03943 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810197:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b0394a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810201:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b0394a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810216:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b0394a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810401:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b03951 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810407:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b03951 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810413:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b03951 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762779.810797:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00010000:00010000:1.0:1789762779.810806:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b03951 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810811:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b03951 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810815:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b03951 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810819:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762779.810821:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b03951 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810825:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b03951 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810831:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff981ec2179d00) 00010000:00010000:1.0:1789762779.810833:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762779.810840:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b03951 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810845:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b03951 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x22:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810856:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b03943 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810860:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b03943 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810864:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b03943 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810867:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762779.810869:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b03943 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810873:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b03943 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810877:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:1.0:1789762779.810879:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762779.810881:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b03943 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762779.810884:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b03943 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762779.810890:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006814c129 reply state 00000000bb11e809 transno 4294967482 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b0394a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762779.810902:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762779.810937:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:13136:x1876702014963712:12345-192.168.203.48@tcp:36:rm.0 Request processed in 872us (958us total) trans 4294967482 rc 0/0 00000100:00100000:1.0:1789762779.810942:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 225 00010000:00010000:3.0:1789762779.811574:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b0394a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762779.811585:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b0394a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762779.811592:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b0394a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762779.811598:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762779.811601:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b0394a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762779.811615:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b0394a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762779.811625:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179800) 00010000:00010000:3.0:1789762779.811628:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762779.811632:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b0394a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762779.811639:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b0394a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762780.087148:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1789762780.664040:0:14384:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 303967950055ns, next at 304111516407ns, rc = 0 00000001:00080000:2.0:1789762780.672206:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967481 is committed 00000001:00080000:2.0:1789762780.672221:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967482 is committed 00000004:00080000:2.0:1789762780.672231:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 17 committed 00000004:00080000:2.0:1789762780.672245:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 18 committed 00000100:00100000:2.0:1789762780.672344:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762780.672362:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfecd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022160000:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762780.672419:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762780.672448:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022160000 00000100:00100000:0.0:1789762780.672455:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022160000 00000100:00080000:2.0:1789762780.672482:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762780.672523:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:0.0:1789762780.672534:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022160000:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1789762780.672558:0:6649:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:14 count 1 00002000:00010000:0.0:1789762780.672588:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 104857600, mtime: 0x6aad9cd2, atime: 0x0, ctime: 0x6aad9cd2, blocks: 0x3200b 00010000:00010000:0.0:1789762780.672594:0:6649:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b03958 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762780.672607:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=303s v=(0 0) 00010000:00010000:0.0:1789762780.672611:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762780.672616:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b03958 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762780.672626:0:6649:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b03958 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762780.672633:0:6649:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b03958 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762780.672638:0:6649:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b03958 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762780.672653:0:6649:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762780.672671:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762780.672677:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b03958 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762780.672687:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aa400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762780.672696:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b03958 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762780.672704:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b03958 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762780.672719:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b03958 lrc: 1/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762780.672724:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b03958 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762780.672737:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762780.672739:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762780.672740:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762780.673062:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022160000:12345-0@lo:6:osp-syn-0-0.0 Request processed in 530us (641us total) trans 4294967792 rc 0/0 00000100:00100000:0.0:1789762780.673069:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000004:00080000:2.0:1789762780.673112:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebfecd500/2, rc 0, transno 4294967792 00000100:00100000:2.0:1789762780.673124:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfecd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022160000:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1789762780.673221:0:14384:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 303977133156ns, next at 291783176961ns, rc = 0 00000004:00080000:1.0:1789762780.674070:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 15 committed 00010000:00010000:3.0:1789762781.111213:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762782.135160:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762782.306355:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762782.306396:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762782.306408:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014963840 00000100:00100000:1.0:1789762782.306421:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014963840 00000100:00100000:2.0:1789762782.306433:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762782.306460:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762782.306486:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 292 00000100:00100000:2.0:1789762782.306506:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762782.306514:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014963840:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762782.306544:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014963968 00000100:00100000:2.0:1789762782.306550:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014963968 00000100:00100000:2.0:1789762782.306595:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014964352 00000100:00100000:2.0:1789762782.306597:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014964352 00000100:00100000:1.0:1789762782.306616:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014963840:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 108us (269us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762782.306628:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 218 00000100:00100000:1.0:1789762782.306633:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 292 00000100:00100000:2.0:1789762782.306642:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014963968:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762782.306694:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014963968:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 56us (297us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762782.306701:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 218 00000100:00100000:2.0:1789762782.306724:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 219 00000100:00100000:2.0:1789762782.306731:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014964352:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762782.306759:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014964352:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 28us (266us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762782.306765:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 219 00000100:00100000:2.0:1789762782.306790:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014964096 00000100:00100000:2.0:1789762782.306795:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014964096 00000100:00080000:1.0:1789762782.306806:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762782.306813:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762782.306836:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014964224 00000100:00100000:1.0:1789762782.306840:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014964224 00000100:00100000:2.0:1789762782.306844:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 107 00000100:00100000:2.0:1789762782.306856:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc2d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014964096:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762782.306880:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 108 00000100:00100000:1.0:1789762782.306888:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014964224:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762782.306895:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc2d3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014964096:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 41us (462us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762782.306902:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 107 00000100:00100000:1.0:1789762782.306924:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014964224:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 37us (463us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762782.306931:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 108 00000100:00080000:1.0:1789762782.306951:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762782.327241:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022160128:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762782.327331:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762782.327364:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022160128 00000100:00100000:0.0:1789762782.327374:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022160128 00000100:00080000:1.0:1789762782.327419:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762782.327445:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 93 00000100:00100000:0.0:1789762782.327456:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022160128:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762782.327512:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439549952 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762782.327576:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022160128:12345-0@lo:13:osp-pre-0-0.0 Request processed in 122us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762782.327584:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 93 00000100:00100000:2.0:1789762782.327614:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022160128:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762782.839276:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82f90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022160256:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762782.839276:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022160384:0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.839280:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022160640:0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.839293:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022160512:0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.839386:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762782.839387:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762782.839387:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762782.839450:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022160384 00000100:00100000:3.0:1789762782.839460:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022160384 00000100:00100000:1.0:1789762782.839468:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022160768:0@lo:400:kworker.0 00000100:00100000:2.0:1789762782.839478:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022160896:0@lo:400:kworker.0 00000100:00100000:2.0:1789762782.839518:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762782.839521:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022160640 00000100:00100000:0.0:1789762782.839527:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022160640 00000100:00100000:3.0:1789762782.839532:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 293 00000100:00100000:1.0:1789762782.839546:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762782.839551:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022160384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762782.839563:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022160256 00000100:00100000:2.0:1789762782.839579:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022160256 00000100:00100000:0.0:1789762782.839587:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022160896 00000100:00100000:0.0:1789762782.839593:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022160896 00000100:00080000:1.0:1789762782.839595:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789762782.839602:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762782.839622:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:3.0:1789762782.839627:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022160384:12345-0@lo:400:kworker.0 Request processed in 82us (242us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762782.839629:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 94 00000100:00100000:0.0:1789762782.839640:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:1.0:1789762782.839641:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022160640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762782.839642:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 293 00000100:00100000:2.0:1789762782.839643:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022160256:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762782.839656:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022160896:12345-0@lo:400:kworker.0 00002000:00000024:2.0:1789762782.839686:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 13631488 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762782.839691:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762782.839712:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022160640:12345-0@lo:400:kworker.0 Request processed in 79us (330us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762782.839718:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022160896:12345-0@lo:400:kworker.0 Request processed in 64us (198us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762782.839726:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:0.0:1789762782.839735:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00100000:3.0:1789762782.839737:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022161024:0@lo:400:kworker.0 00000100:00100000:2.0:1789762782.839750:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022160256:12345-0@lo:13:osp-pre-1-0.0 Request processed in 109us (362us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762782.839756:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022160512 00000100:00100000:1.0:1789762782.839760:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022160512 00000100:00100000:2.0:1789762782.839760:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 94 00000100:00100000:3.0:1789762782.839776:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762782.839778:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022160768 00000100:00100000:0.0:1789762782.839784:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022160768 00000100:00100000:1.0:1789762782.839791:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:1.0:1789762782.839803:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022160512:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.839828:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022161024 00000100:00100000:2.0:1789762782.839834:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 109 00000100:00100000:0.0:1789762782.839835:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022161024 00000100:00100000:2.0:1789762782.839849:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022160768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762782.839854:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022160512:12345-0@lo:400:kworker.0 Request processed in 56us (166us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762782.839862:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00100000:3.0:1789762782.839868:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022160896:0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.839869:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 110 00000100:00100000:0.0:1789762782.839885:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022161024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762782.839890:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022160768:12345-0@lo:400:kworker.0 Request processed in 43us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762782.839898:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022160512:0@lo:400:kworker.0 00000100:00100000:2.0:1789762782.839901:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 109 00000100:00100000:3.0:1789762782.839909:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022160384:0@lo:400:kworker.0 00000100:00100000:1.0:1789762782.839944:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022161024:0@lo:400:kworker.0 00000100:00100000:0.0:1789762782.839946:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022161024:12345-0@lo:400:kworker.0 Request processed in 64us (171us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762782.839954:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82f90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022160256:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762782.839962:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 110 00000100:00100000:2.0:1789762782.839975:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022160768:0@lo:400:kworker.0 00000100:00100000:1.0:1789762782.839979:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022160640:0@lo:400:kworker.0 00010000:00010000:0.0:1789762783.159191:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762784.183263:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762785.207275:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762785.420954:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967792 is committed 00010000:00010000:0.0:1789762786.231302:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762787.255232:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762787.426613:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762787.426693:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762787.426735:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762787.426766:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014964736 00000100:00100000:3.0:1789762787.426781:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014964736 00000100:00100000:1.0:1789762787.426783:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014964480 00000100:00100000:1.0:1789762787.426792:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014964480 00000100:00080000:2.0:1789762787.426868:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762787.426886:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014964608 00000100:00100000:3.0:1789762787.426899:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 111 00000100:00100000:3.0:1789762787.426921:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014964736:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762787.426931:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014964608 00000100:00100000:1.0:1789762787.426938:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 294 00000100:00100000:1.0:1789762787.426951:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702014964480:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:2.0:1789762787.426970:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762787.427041:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014964736:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 125us (341us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762787.427052:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 223 00000100:00100000:1.0:1789762787.427058:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702014964480:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 89us (435us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762787.427066:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 111 00000100:00100000:1.0:1789762787.427145:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 294 00000100:00100000:0.0:1789762787.427153:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014964608:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762787.427258:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014964608:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 185us (519us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762787.427270:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 223 00000100:00100000:2.0:1789762787.430002:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762787.430120:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014964864 00000100:00100000:0.0:1789762787.430127:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014964864 00000100:00080000:1.0:1789762787.430183:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762787.430185:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762787.430212:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 150 00000100:00100000:0.0:1789762787.430232:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2362:x1876702014964864:12345-192.168.203.48@tcp:9:kworker.0 00000100:00100000:2.0:1789762787.430236:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014964992 00000100:00100000:2.0:1789762787.430244:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014964992 00000020:00000080:0.0:1789762787.430260:0:6648:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x416193e012b0311d 00000100:00080000:0.0:1789762787.430275:0:6648:0:(import.c:65:import_set_state_nolock()) ffff981ebf588800 : changing import state from FULL to CLOSED 00000100:00080000:1.0:1789762787.430293:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762787.430306:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 224 00010000:00010000:3.0:1789762787.430307:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762787.430317:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014964992:12345-192.168.203.48@tcp:400:kworker.0 00000001:00000004:0.0:1789762787.430324:0:6648:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967380 00000100:00100000:2.0:1789762787.430389:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702014964992:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 73us (208us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762787.430410:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 224 00000100:00100000:0.0:1789762787.430664:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+2:2362:x1876702014964864:12345-192.168.203.48@tcp:9:kworker.0 Request processed in 441us (660us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762787.430672:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 150 00000100:00100000:2.0:1789762787.447273:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789762787.447301:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad81500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022161152:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762787.447363:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762787.447526:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022161152 00000100:00100000:0.0:1789762787.447535:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022161152 00000100:00080000:1.0:1789762787.447585:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762787.447586:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 95 00000100:00100000:0.0:1789762787.447602:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022161152:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762787.447628:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439549952 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762787.447688:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022161152:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (325us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762787.447696:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 95 00000100:00100000:2.0:1789762787.447718:0:3274:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967792 gen 2 00000004:00080000:2.0:1789762787.447724:0:3274:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebfecd500, transno 4294967792 00000100:00100000:2.0:1789762787.447745:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad81500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022161152:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762787.959275:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad82300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022161280:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.959294:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82f92300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022161408:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762787.959313:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022161536:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.959388:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762787.959400:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762787.959401:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762787.959478:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022161408 00000100:00100000:1.0:1789762787.959500:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022161408 00000100:00100000:2.0:1789762787.959524:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad81c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022161792:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.959526:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022161920:0@lo:400:kworker.0 00000100:00100000:0.0:1789762787.959531:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad81880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022162048:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.959579:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762787.959589:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 96 00000100:00100000:1.0:1789762787.959608:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022161408:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762787.959617:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789762787.959627:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762787.959633:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762787.959639:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad80700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022161664:0@lo:400:kworker.0 00002000:00000024:1.0:1789762787.959681:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762787.959694:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762787.959747:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022161536 00000100:00100000:2.0:1789762787.959752:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022161280 00000100:00100000:0.0:1789762787.959756:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022161536 00000100:00100000:2.0:1789762787.959759:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022161280 00000100:00100000:3.0:1789762787.959771:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022161792 00000100:00100000:1.0:1789762787.959773:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022161408:12345-0@lo:13:osp-pre-1-0.0 Request processed in 170us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762787.959777:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022161792 00000100:00100000:1.0:1789762787.959792:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 96 00000100:00100000:2.0:1789762787.959833:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 295 00000100:00080000:1.0:1789762787.959841:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762787.959845:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022162048 00000100:00100000:2.0:1789762787.959852:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022161280:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.959853:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022162048 00000100:00100000:0.0:1789762787.959854:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022161920 00000100:00100000:0.0:1789762787.959865:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022161920 00000100:00100000:1.0:1789762787.959866:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022161664 00000100:00100000:1.0:1789762787.959875:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022161664 00000100:00100000:3.0:1789762787.959913:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 112 00000100:00100000:3.0:1789762787.959926:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022161792:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.959941:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022161280:12345-0@lo:400:kworker.0 Request processed in 93us (553us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762787.959953:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:2.0:1789762787.959968:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 295 00000100:00100000:1.0:1789762787.959975:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:0.0:1789762787.959981:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022161536:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.960007:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022161792:12345-0@lo:400:kworker.0 Request processed in 79us (427us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762787.960012:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022161920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.960020:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 112 00000100:00100000:2.0:1789762787.960024:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:2.0:1789762787.960036:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022161664:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.960051:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:3.0:1789762787.960063:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022162048:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762787.960092:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022161536:12345-0@lo:400:kworker.0 Request processed in 121us (706us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762787.960102:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00100000:2.0:1789762787.960127:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022161664:12345-0@lo:400:kworker.0 Request processed in 93us (439us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762787.960136:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022161920:12345-0@lo:400:kworker.0 Request processed in 143us (518us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762787.960136:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00100000:1.0:1789762787.960155:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:3.0:1789762787.960166:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022162048:12345-0@lo:400:kworker.0 Request processed in 106us (555us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762787.960176:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00080000:1.0:1789762787.960197:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762787.960211:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad81880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022162048:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.960225:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad80700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022161664:0@lo:400:kworker.0 00000100:00100000:0.0:1789762787.960234:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022161536:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.960261:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad81c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022161792:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.960266:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022161920:0@lo:400:kworker.0 00000100:00100000:2.0:1789762787.960293:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad82300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022161280:0@lo:400:kworker.0 00000100:00100000:3.0:1789762787.960314:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82f92300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022161408:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789762788.279227:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:1.0:1789762788.992660:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:1.0:1789762788.992666:0:7605:0:(genops.c:885:class_export_put()) final put ffff981ebf589800/f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:2.0:1789762788.992702:0:8:0:(genops.c:844:class_export_destroy()) destroying export ffff981ebf589800/f1bd28de-ee78-4d18-bad3-fa7cec968893 for lustre-OST0001 00000020:00000080:2.0:1789762788.992755:0:8:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff981ebf588800 for lustre-OST0001 00010000:00010000:0.0:1789762789.303267:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762790.327235:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762791.351267:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762792.375245:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762792.546487:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762792.546537:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762792.546553:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014965248 00000100:00100000:0.0:1789762792.546566:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014965248 00000100:00100000:2.0:1789762792.546571:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762792.546613:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014965120 00000100:00100000:1.0:1789762792.546624:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014965120 00000100:00100000:2.0:1789762792.546624:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762792.546639:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014965376 00000100:00100000:0.0:1789762792.546647:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 228 00000100:00100000:3.0:1789762792.546647:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014965376 00000100:00100000:2.0:1789762792.546667:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014965504 00000100:00100000:0.0:1789762792.546675:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014965248:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762792.546676:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014965504 00000100:00100000:1.0:1789762792.546700:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 296 00000100:00100000:3.0:1789762792.546707:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 114 00000100:00100000:1.0:1789762792.546714:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014965120:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762792.546719:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014965376:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762792.546727:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 229 00000100:00100000:2.0:1789762792.546738:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014965504:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762792.546771:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014965248:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 104us (289us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762792.546772:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014965376:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 49us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762792.546790:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 228 00000100:00100000:3.0:1789762792.546792:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 114 00000100:00100000:1.0:1789762792.546804:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014965120:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 85us (255us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762792.546806:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014965504:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 67us (178us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762792.546818:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 296 00000100:00100000:2.0:1789762792.546823:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 229 00000100:00080000:0.0:1789762792.546846:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762792.546853:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762792.567296:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022162176:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762792.567352:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762792.567401:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022162176 00000100:00100000:0.0:1789762792.567408:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022162176 00000100:00080000:1.0:1789762792.567470:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762792.567491:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 97 00000100:00100000:0.0:1789762792.567517:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022162176:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762792.567549:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439549952 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762792.567601:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022162176:12345-0@lo:13:osp-pre-0-0.0 Request processed in 89us (247us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762792.567610:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 97 00000100:00100000:3.0:1789762792.567644:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad80380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022162176:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762793.079292:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82f91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022162304:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762793.079310:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022162432:0@lo:400:kworker.0 00000100:00100000:3.0:1789762793.079343:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022162560:0@lo:400:kworker.0 00000100:00100000:2.0:1789762793.079387:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022162688:0@lo:400:kworker.0 00000100:00100000:1.0:1789762793.079395:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762793.079402:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762793.079425:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762793.079499:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762793.079503:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022162816:0@lo:400:kworker.0 00000100:00100000:0.0:1789762793.079506:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022162944:0@lo:400:kworker.0 00000100:00100000:3.0:1789762793.079506:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022163072:0@lo:400:kworker.0 00000100:00100000:1.0:1789762793.079582:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762793.079583:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022162304 00000100:00100000:3.0:1789762793.079590:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022162560 00000100:00100000:0.0:1789762793.079591:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762793.079597:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022162304 00000100:00100000:3.0:1789762793.079601:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022162560 00000100:00100000:3.0:1789762793.079662:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:1.0:1789762793.079675:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022162688 00000100:00100000:0.0:1789762793.079677:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022162432 00000100:00100000:0.0:1789762793.079684:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022162432 00000100:00100000:1.0:1789762793.079684:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022162688 00000100:00100000:2.0:1789762793.079686:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 98 00000100:00100000:3.0:1789762793.079699:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022162560:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762793.079712:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022162304:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762793.079733:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:1.0:1789762793.079741:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022162944 00000100:00100000:1.0:1789762793.079748:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022162944 00000100:00100000:0.0:1789762793.079749:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022162432:12345-0@lo:400:kworker.0 00002000:00000024:2.0:1789762793.079770:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762793.079779:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022162560:12345-0@lo:400:kworker.0 Request processed in 84us (371us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762793.079790:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:1.0:1789762793.079802:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 231 00000100:00100000:0.0:1789762793.079817:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022162432:12345-0@lo:400:kworker.0 Request processed in 70us (425us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762793.079818:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022162688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762793.079824:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:0.0:1789762793.079837:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:3.0:1789762793.079842:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022162944:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762793.079855:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad83b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022162304:12345-0@lo:13:osp-pre-1-0.0 Request processed in 146us (470us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762793.079864:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 98 00000100:00100000:3.0:1789762793.079903:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022162944:12345-0@lo:400:kworker.0 Request processed in 67us (327us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762793.079912:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00100000:1.0:1789762793.079919:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022162688:12345-0@lo:400:kworker.0 Request processed in 102us (417us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762793.079934:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 231 00000100:00100000:2.0:1789762793.079939:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022162816 00000100:00100000:0.0:1789762793.079945:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022162944:0@lo:400:kworker.0 00000100:00100000:2.0:1789762793.079946:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022162816 00000100:00100000:0.0:1789762793.079970:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022162432:0@lo:400:kworker.0 00000100:00080000:3.0:1789762793.079974:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789762793.079983:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762793.080016:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:1.0:1789762793.080025:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82f91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022162304:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762793.080026:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022162816:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762793.080055:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022162688:0@lo:400:kworker.0 00000100:00100000:3.0:1789762793.080063:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762793.080149:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022163072 00000100:00100000:2.0:1789762793.080152:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad82d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022162816:12345-0@lo:400:kworker.0 Request processed in 120us (575us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762793.080153:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022162816:0@lo:400:kworker.0 00000100:00100000:0.0:1789762793.080159:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022163072 00000100:00100000:3.0:1789762793.080165:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022162560:0@lo:400:kworker.0 00000100:00100000:2.0:1789762793.080176:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00100000:2.0:1789762793.080211:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 116 00000100:00100000:2.0:1789762793.080224:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022163072:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762793.080265:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022163072:12345-0@lo:400:kworker.0 Request processed in 46us (211us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762793.080273:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 116 00000100:00100000:3.0:1789762793.080313:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022163072:0@lo:400:kworker.0 00010000:00010000:0.0:1789762793.399223:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762794.423286:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762795.447246:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762796.471266:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762797.495222:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762797.666244:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762797.666300:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762797.666310:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014965760 00000100:00100000:2.0:1789762797.666337:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762797.666341:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014965760 00000100:00100000:1.0:1789762797.666351:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014965632 00000100:00100000:1.0:1789762797.666358:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014965632 00000100:00100000:2.0:1789762797.666374:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762797.666396:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014965888 00000100:00100000:3.0:1789762797.666403:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014965888 00000100:00100000:0.0:1789762797.666431:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014966016 00000100:00100000:1.0:1789762797.666436:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 298 00000100:00100000:0.0:1789762797.666437:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014966016 00000100:00080000:2.0:1789762797.666437:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762797.666447:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762797.666451:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 117 00000100:00100000:1.0:1789762797.666457:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014965632:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762797.666468:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014965888:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762797.666476:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 233 00000100:00100000:2.0:1789762797.666485:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 234 00000100:00100000:0.0:1789762797.666490:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014965760:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762797.666503:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014966016:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762797.666542:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702014965632:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 93us (242us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762797.666551:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014965760:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 59us (311us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762797.666552:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014965888:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 81us (205us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762797.666554:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702014966016:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 55us (176us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762797.666570:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 233 00000100:00100000:3.0:1789762797.666577:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 117 00000100:00100000:1.0:1789762797.666578:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 298 00000100:00100000:2.0:1789762797.666592:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 234 00000100:00100000:0.0:1789762797.687276:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad83800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022163200:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762797.687323:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762797.687388:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022163200 00000100:00100000:2.0:1789762797.687396:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022163200 00000100:00080000:3.0:1789762797.687425:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762797.687447:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 99 00000100:00100000:2.0:1789762797.687461:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022163200:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762797.687497:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 439549952 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762797.687548:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022163200:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762797.687557:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 99 00000100:00100000:0.0:1789762797.687585:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad83800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022163200:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762798.199227:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad82a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022163328:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762798.199309:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762798.199408:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022163328 00000100:00100000:0.0:1789762798.199417:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022163328 00000100:00100000:0.0:1789762798.199477:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 100 00000100:00100000:0.0:1789762798.199487:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022163328:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789762798.199520:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762798.199573:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022163328:12345-0@lo:13:osp-pre-1-0.0 Request processed in 88us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762798.199579:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 100 00000100:00080000:1.0:1789762798.199589:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762798.199632:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad82a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022163328:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762798.200293:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022163584:0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.200337:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762798.200362:0:3272:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1789762798.200370:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022163456:0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.200430:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762798.200458:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022163968:0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.200486:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762798.200509:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022163712:0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.200527:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762798.200548:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789762798.200553:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022163840:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.200584:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022163456 00000100:00100000:3.0:1789762798.200589:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762798.200591:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022163456 00000100:00100000:0.0:1789762798.200626:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022163584 00000100:00100000:0.0:1789762798.200632:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022163584 00000100:00100000:3.0:1789762798.200634:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022163968 00000100:00100000:1.0:1789762798.200638:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 299 00000100:00100000:3.0:1789762798.200642:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022163968 00000100:00100000:1.0:1789762798.200650:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022163456:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762798.200662:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022163712 00000100:00100000:0.0:1789762798.200666:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022163712 00000100:00100000:3.0:1789762798.200687:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:3.0:1789762798.200697:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022163584:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.200699:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022163456:12345-0@lo:400:kworker.0 Request processed in 51us (267us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762798.200712:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:1.0:1789762798.200712:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 299 00000100:00100000:0.0:1789762798.200729:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022163968:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.200755:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022163584:12345-0@lo:400:kworker.0 Request processed in 61us (417us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762798.200764:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00100000:1.0:1789762798.200765:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022163840 00000100:00100000:1.0:1789762798.200770:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022163840 00000100:00100000:0.0:1789762798.200775:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022163968:12345-0@lo:400:kworker.0 Request processed in 50us (287us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762798.200784:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00000100:00100000:3.0:1789762798.200793:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:3.0:1789762798.200806:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022163712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.200809:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 118 00000100:00100000:1.0:1789762798.200821:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022163840:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.200856:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022163712:12345-0@lo:400:kworker.0 Request processed in 54us (327us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762798.200863:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1789762798.200874:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022163840:0@lo:400:kworker.0 00000100:00100000:1.0:1789762798.200876:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022163840:12345-0@lo:400:kworker.0 Request processed in 53us (285us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762798.200889:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 118 00000100:00100000:0.0:1789762798.200890:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022163712:0@lo:400:kworker.0 00000100:00080000:3.0:1789762798.200891:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762798.200917:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022163456:0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.200937:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022163968:0@lo:400:kworker.0 00000100:00100000:3.0:1789762798.200945:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022163584:0@lo:400:kworker.0 00010000:00010000:1.0:1789762798.520161:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762799.240926:0:14900:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.48@tcp 00000100:00100000:3.0:1789762799.240951:0:14900:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:3.0:1789762799.240963:0:14900:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762799.240970:0:14900:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.48@tcp 00000100:00100000:3.0:1789762799.240979:0:14900:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:3.0:1789762799.240983:0:14900:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981ebc5ded00 rate 20 gen 0 00000100:00100000:3.0:1789762799.240987:0:14900:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762799.240990:0:14900:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: nid, str: 0@lo 192.168.203.48@tcp 00000100:00100000:3.0:1789762799.240996:0:14900:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:3.0:1789762799.240999:0:14900:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981ebc5dee00 rate 20 gen 0 00000100:00100000:2.0:1789762799.252680:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.252733:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014966144 00000100:00100000:0.0:1789762799.252748:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014966144 00000100:00080000:3.0:1789762799.252796:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762799.252833:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 226 00000100:00100000:0.0:1789762799.252848:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:13614:x1876702014966144:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:0.0:1789762799.252871:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762799.252894:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b0395f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4d4 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.252966:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03966 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.252980:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03966 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.252993:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03966 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762799.253039:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762799.253043:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762799.253062:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03966 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.253122:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b0395f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4d4 expref: 7 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.253140:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03966 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4d4 expref: 6 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.253150:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3300) 00010000:00010000:0.0:1789762799.253152:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762799.253156:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aac00, rc 0) 00000100:00100000:0.0:1789762799.253231:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:13614:x1876702014966144:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 387us (557us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762799.253239:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 226 00000100:00100000:2.0:1789762799.254686:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.254724:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014966272 00000100:00100000:3.0:1789762799.254731:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014966272 00000100:00100000:2.0:1789762799.254736:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.254773:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014966528 00000100:00100000:0.0:1789762799.254778:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014966528 00000100:00100000:0.0:1789762799.254808:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 227 00000100:00100000:3.0:1789762799.254821:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 228 00000100:00100000:0.0:1789762799.254822:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2363:x1876702014966272:12345-192.168.203.48@tcp:41:mkdir.0 00000100:00100000:3.0:1789762799.254839:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:13614:x1876702014966528:12345-192.168.203.48@tcp:36:mkdir.0 00000004:00000024:0.0:1789762799.254858:0:6611:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00010000:00010000:3.0:1789762799.254874:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762799.254891:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03966 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4d4 expref: 7 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762799.254900:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2363:x1876702014966272:12345-192.168.203.48@tcp:41:mkdir.0 Request processed in 83us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762799.254908:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 227 00010000:00010000:3.0:1789762799.254908:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3300) 00010000:00010000:3.0:1789762799.254924:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762799.255237:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b0396d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.255246:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b0396d lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.255258:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b0396d lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.255269:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03974 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.255275:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03974 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.255280:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03974 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762799.256445:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00010000:00010000:3.0:1789762799.256458:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b0396d lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.256466:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b0396d lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.256475:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b0396d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.256484:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.256489:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b0396d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.256499:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b0396d lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.256506:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46f00) 00010000:00010000:3.0:1789762799.256509:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.256519:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b0396d lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.256524:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b0396d lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762799.256535:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000035b57319 reply state 000000008e099f3f transno 4294967483 ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03974 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762799.256543:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762799.256588:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:13614:x1876702014966528:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 1755us (1851us total) trans 4294967483 rc 0/0 00000100:00100000:3.0:1789762799.256594:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 228 00010000:00010000:0.0:1789762799.257033:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03974 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.257041:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03974 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.257046:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03974 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.257057:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762799.257059:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03974 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.257064:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03974 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.257071:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e83a46500) 00010000:00010000:0.0:1789762799.257074:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762799.257076:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03974 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.257080:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03974 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762799.261627:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.261693:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014966656 00000100:00100000:0.0:1789762799.261699:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014966656 00000100:00100000:0.0:1789762799.261734:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 229 00000100:00100000:0.0:1789762799.261743:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:13615:x1876702014966656:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762799.261754:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762799.261764:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b0397b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4db expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.261793:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b03982 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.261801:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b03982 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.261818:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b03982 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762799.261833:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762799.261835:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762799.261846:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b03982 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.261861:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b0397b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4db expref: 8 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.261872:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b03982 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4db expref: 7 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.261878:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3000) 00010000:00010000:0.0:1789762799.261881:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762799.261883:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880ab200, rc 0) 00000100:00100000:0.0:1789762799.261913:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:13615:x1876702014966656:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 173us (289us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762799.261919:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 229 00000100:00100000:2.0:1789762799.263156:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.263206:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014966912 00000100:00100000:3.0:1789762799.263211:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014966912 00000100:00100000:3.0:1789762799.263246:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 230 00000100:00100000:3.0:1789762799.263254:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81397b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:13615:x1876702014966912:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762799.263262:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.263263:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1789762799.263278:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b03982 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4db expref: 7 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263290:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3000) 00010000:00010000:3.0:1789762799.263297:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762799.263305:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b03989 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4e2 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263394:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03990 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263419:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03990 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263429:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03990 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263446:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03997 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263456:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03997 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263464:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03997 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263543:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b0399e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263555:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b0399e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263564:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b0399e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762799.263574:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00010000:00010000:3.0:1789762799.263590:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b0399e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263599:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b0399e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263606:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b0399e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263621:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.263626:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b0399e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263634:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b0399e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263645:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46200) 00010000:00010000:3.0:1789762799.263649:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.263653:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b0399e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263660:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b0399e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263678:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03990 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263686:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03990 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263693:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03990 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263699:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.263703:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03990 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263710:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03990 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263734:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46f00) 00010000:00010000:3.0:1789762799.263737:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.263741:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03990 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263748:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03990 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263760:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03997 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263767:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03997 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263775:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03997 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263781:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.263785:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03997 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263793:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03997 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263803:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e83a46000) 00010000:00010000:3.0:1789762799.263806:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.263809:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03997 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263816:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03997 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263864:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b03989 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4e2 expref: 7 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263875:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46f00) 00010000:00010000:3.0:1789762799.263879:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.263884:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b03989 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4e2 expref: 7 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.263898:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa33600, rc 0) 00000100:00100000:3.0:1789762799.263940:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81397b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:13615:x1876702014966912:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 685us (783us total) trans 0 rc 301/301 00000100:00100000:3.0:1789762799.263950:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 230 00000100:00100000:2.0:1789762799.264772:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.264803:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014967040 00000100:00100000:3.0:1789762799.264808:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014967040 00000100:00100000:3.0:1789762799.264836:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 231 00000100:00100000:3.0:1789762799.264845:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81396d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:13615:x1876702014967040:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762799.264853:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.264863:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b039a5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4e9 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.264887:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33400/0x416193e012b039ac lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.264894:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33400/0x416193e012b039ac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.264900:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33400/0x416193e012b039ac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762799.264921:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762799.264923:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762799.264932:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33400/0x416193e012b039ac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.264947:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b039a5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4e9 expref: 8 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.264958:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33400/0x416193e012b039ac lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4e9 expref: 7 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.264965:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:3.0:1789762799.264969:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762799.264971:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa33400, rc 0) 00000100:00100000:3.0:1789762799.265012:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81396d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:13615:x1876702014967040:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 168us (240us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762799.265020:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 231 00000100:00100000:2.0:1789762799.265887:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.265929:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014967296 00000100:00100000:3.0:1789762799.265935:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014967296 00000100:00100000:3.0:1789762799.265974:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 232 00000100:00100000:3.0:1789762799.265984:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81397800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:13615:x1876702014967296:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762799.265994:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.266005:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b039b3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4f0 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266043:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039ba lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266066:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039ba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266076:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039ba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266121:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039c1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266131:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039c1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266140:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039c1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762799.266202:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00010000:00010000:3.0:1789762799.266281:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039ba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266289:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039ba lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266296:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039ba lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266302:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.266306:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039ba lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266313:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039ba lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266336:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:3.0:1789762799.266339:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.266343:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039ba lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266351:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039ba lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266363:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039c1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266380:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039c1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266385:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039c1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266388:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.266390:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039c1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266394:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039c1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266400:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e83a46d00) 00010000:00010000:3.0:1789762799.266401:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.266403:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039c1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266407:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039c1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266437:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b039b3 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4f0 expref: 9 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266444:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:3.0:1789762799.266446:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762799.266448:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b039b3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4f0 expref: 9 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.266454:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa33a00, rc 0) 00000100:00100000:3.0:1789762799.266480:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81397800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:13615:x1876702014967296:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 499us (594us total) trans 4294967484 rc 301/301 00000100:00100000:3.0:1789762799.266486:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 232 00000100:00100000:1.0:1789762799.267258:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.267292:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014967552 00000100:00100000:3.0:1789762799.267296:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014967552 00000100:00100000:3.0:1789762799.267325:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 233 00000100:00100000:3.0:1789762799.267332:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad82680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:13615:x1876702014967552:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:3.0:1789762799.267362:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b039cf lrc: 3/0,1 mode: --/PW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.267382:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b039cf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.267388:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b039cf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762799.267487:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d2ef856b reply state 00000000383ebd83 transno 4294967485 ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b039cf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762799.267509:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762799.267516:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00000100:00100000:3.0:1789762799.267570:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad82680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13615:x1876702014967552:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 238us (312us total) trans 4294967485 rc 0/0 00000100:00100000:3.0:1789762799.267577:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 233 00010000:00010000:3.0:1789762799.268111:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b039cf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.268119:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b039cf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.268124:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b039cf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.268130:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.268134:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b039cf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.268143:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b039cf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.268156:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46500) 00000100:00100000:2.0:1789762799.268158:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:3.0:1789762799.268164:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789762799.268170:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b039cf lrc: 2/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.268178:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b039cf lrc: 0/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762799.268187:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014967808 00000100:00100000:0.0:1789762799.268192:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014967808 00000100:00100000:0.0:1789762799.268226:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 68 00000100:00100000:0.0:1789762799.268237:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81396a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13615:x1876702014967808:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:0.0:1789762799.268354:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81396a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13615:x1876702014967808:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 120us (200us total) trans 4294967486 rc 0/0 00000100:00100000:0.0:1789762799.268360:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 68 00000100:00100000:2.0:1789762799.271977:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.272015:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014967936 00000100:00100000:3.0:1789762799.272025:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014967936 00000100:00100000:3.0:1789762799.272058:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 234 00000100:00100000:3.0:1789762799.272068:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81396680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13616:x1876702014967936:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:3.0:1789762799.272080:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.272127:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b039d6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4f7 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272162:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039dd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272171:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039dd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272180:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039dd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272192:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039e4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272200:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039e4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272208:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039e4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272266:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b039eb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272275:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b039eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272283:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b039eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762799.272307:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00000004:00200000:3.0:1789762799.272310:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762799.272322:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b039eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272330:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039dd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272336:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039dd lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272344:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039dd lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272351:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.272355:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039dd lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272362:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039dd lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272392:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:3.0:1789762799.272396:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.272400:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039dd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272406:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b039dd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272417:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039e4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272434:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039e4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272441:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039e4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272447:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.272451:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039e4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272463:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039e4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272473:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e83a46d00) 00010000:00010000:3.0:1789762799.272477:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.272480:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039e4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272488:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b039e4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272516:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b039d6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4f7 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272529:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b039eb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4f7 expref: 11 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.272535:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46f00) 00010000:00010000:3.0:1789762799.272538:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762799.272545:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa32000, rc 0) 00000100:00100000:3.0:1789762799.272576:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81396680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13616:x1876702014967936:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 511us (601us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762799.272583:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 234 00000100:00100000:2.0:1789762799.273502:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.273533:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014968192 00000100:00100000:3.0:1789762799.273537:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014968192 00000100:00100000:3.0:1789762799.273564:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 235 00000100:00100000:3.0:1789762799.273571:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13616:x1876702014968192:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:3.0:1789762799.273592:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762799.273602:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b039eb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4f7 expref: 11 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.273611:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46f00) 00010000:00010000:3.0:1789762799.273617:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762799.273627:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b039f2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.273633:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b039f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.273638:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b039f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762799.273742:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005045002e reply state 000000001ce854ef transno 4294967487 ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b039f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762799.273748:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762799.273752:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00000100:00100000:3.0:1789762799.273791:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:13616:x1876702014968192:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 221us (293us total) trans 4294967487 rc 0/0 00000100:00100000:3.0:1789762799.273805:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 235 00010000:00010000:0.0:1789762799.274257:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b039f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.274264:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b039f2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.274269:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b039f2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.274272:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762799.274274:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b039f2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.274279:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b039f2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.274285:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46000) 00010000:00010000:0.0:1789762799.274287:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762799.274289:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b039f2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.274294:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b039f2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762799.287878:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.287965:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014968320 00000100:00100000:3.0:1789762799.287975:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014968320 00000100:00100000:3.0:1789762799.288030:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 236 00000100:00100000:3.0:1789762799.288042:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81397100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13620:x1876702014968320:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762799.288061:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.288078:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b039f9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd4fe expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288153:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03a00 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288169:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03a00 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288181:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03a00 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288197:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a07 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288206:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a07 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288214:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a07 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288310:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b03a0e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288332:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b03a0e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288340:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b03a0e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762799.288378:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00000004:00200000:3.0:1789762799.288383:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762799.288404:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b03a0e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288416:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03a00 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288424:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03a00 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288432:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03a00 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288440:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.288445:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03a00 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288453:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03a00 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288463:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:3.0:1789762799.288465:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.288468:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03a00 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288482:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03a00 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288491:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a07 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288495:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a07 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288500:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a07 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288503:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.288505:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a07 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288509:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a07 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288513:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e83a46f00) 00010000:00010000:3.0:1789762799.288515:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.288516:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a07 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288520:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a07 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288543:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b039f9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4fe expref: 13 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288554:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b03a0e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4fe expref: 12 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.288566:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:3.0:1789762799.288568:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789762799.288569:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa32a00, rc 0) 00000100:00100000:3.0:1789762799.288616:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81397100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:13620:x1876702014968320:12345-192.168.203.48@tcp:101:dd.500 Request processed in 577us (743us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762799.288622:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 236 00000100:00100000:2.0:1789762799.289530:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.289561:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014968576 00000100:00100000:3.0:1789762799.289566:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014968576 00000100:00100000:3.0:1789762799.289591:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 237 00000100:00100000:3.0:1789762799.289598:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:13620:x1876702014968576:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762799.289605:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.289612:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a15 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd505 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.289672:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a1c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.289680:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a1c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.289686:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a1c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.290911:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a23 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.290930:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a23 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.290958:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a23 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762799.291985:0:5804:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xf:0x0], index 0: 0 00000004:00080000:3.0:1789762799.291995:0:5804:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xa:0x0], index 1: 0 00010000:00010000:3.0:1789762799.292180:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b03a2a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292188:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b03a2a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292194:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b03a2a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292219:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a1c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292225:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a1c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292229:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a1c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292233:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.292236:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a1c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292241:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a1c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292249:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:3.0:1789762799.292257:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.292261:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a1c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292266:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a1c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762799.292277:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000531b5055 reply state 00000000471c7816 transno 4294967488 ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a23 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762799.292284:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789762799.292313:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a15 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd505 expref: 15 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292324:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b03a2a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd505 expref: 14 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292329:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff981e83a46200) 00010000:00010000:3.0:1789762799.292331:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762799.292333:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa32200, rc 0) 00000100:00100000:3.0:1789762799.292413:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:13620:x1876702014968576:12345-192.168.203.48@tcp:101:dd.500 Request processed in 2814us (2882us total) trans 4294967488 rc 0/0 00000100:00100000:3.0:1789762799.292420:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 237 00010000:00010000:3.0:1789762799.292963:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a23 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292970:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a23 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292974:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a23 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292984:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.292986:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a23 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292991:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a23 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.292997:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x77289514 (ffff981e83a46500) 00010000:00010000:3.0:1789762799.292999:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.293001:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a23 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.293005:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a23 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762799.294669:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.294736:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014968704 00000100:00100000:3.0:1789762799.294743:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014968704 00000100:00080000:1.0:1789762799.294794:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762799.294795:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 151 00000100:00100000:3.0:1789762799.294812:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:13620:x1876702014968704:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762799.294823:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.294835:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000eb30978/0x416193e012b03a38 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd50c expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762799.294853:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xf:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762799.294867:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=322s v=(0 0) 00010000:00010000:3.0:1789762799.294871:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762799.294878:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000eb30978/0x416193e012b03a38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd50c expref: 6 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.294900:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000eb30978/0x416193e012b03a38 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd50c expref: 6 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.294908:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa32000, rc 0) 00000100:00100000:3.0:1789762799.294936:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:13620:x1876702014968704:12345-192.168.203.48@tcp:101:dd.500 Request processed in 127us (269us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762799.294942:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 151 00000100:00100000:1.0:1789762799.301949:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762799.301996:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014968832 00000100:00100000:2.0:1789762799.302003:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014968832 00002000:00100000:2.0:1789762799.302045:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xf:0x0] (0->1048575) req@ffff981ebad81180 x1876702014968832/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762799.302070:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad81180 with x1876702014968832 ext(0->1048575) 00002000:00010000:2.0:1789762799.302075:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad81180 00000100:00100000:2.0:1789762799.302119:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258640781805 time 258640846451 state 0x0 ref 1 00000100:00100000:2.0:1789762799.302127:0:6654:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 6: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804f00 gen 0 rate 20 depth 3 token 0 deadline 258640781805 time 258640846451 state 0x0 ref 1 00000100:00100000:2.0:1789762799.302152:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 3 deadline 322656057905 time 322606057905 state 0x1 ref 1 00000100:00100000:2.0:1789762799.302173:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 322656057905 time 322606081260 state 0x0 ref 1 00000100:00100000:2.0:1789762799.302181:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014968832:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762799.307828:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014968832:12345-192.168.203.48@tcp:4:dd.500 Request processed in 5643us (5877us total) trans 4294967793 rc 0/0 00000100:00100000:0.0:1789762799.307838:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 461 00000100:00100000:0.0:1789762799.307846:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 322656057905 time 322606081260 state 0x2 ref 0 00002000:00100000:0.0:1789762799.307857:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xf:0x0] (0->1048575) req@ffff981ebad81180 x1876702014968832/t4294967793(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762799.307878:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad81180 with x1876702014968832 ext(0->1048575) 00002000:00010000:0.0:1789762799.307882:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad81180 00000100:00100000:1.0:1789762799.308577:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.308616:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014968960 00000100:00100000:3.0:1789762799.308623:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014968960 00002000:00100000:3.0:1789762799.308651:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0xf:0x0] (1048576->2097151) req@ffff981ebad83100 x1876702014968960/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.308661:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad83100 with x1876702014968960 ext(1048576->2097151) 00002000:00010000:3.0:1789762799.308667:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad83100 00000100:00100000:3.0:1789762799.308683:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 322656057905 time 322606081260 state 0x0 ref 1 00000100:00100000:3.0:1789762799.308698:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 322656081260 time 322606081260 state 0x1 ref 1 00000100:00100000:3.0:1789762799.308726:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 322656081260 time 322612634496 state 0x0 ref 1 00000100:00100000:3.0:1789762799.308737:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702014968960:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762799.311309:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702014968960:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2573us (2731us total) trans 4294967794 rc 0/0 00000100:00100000:3.0:1789762799.311326:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 462 00000100:00100000:3.0:1789762799.311334:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 322656081260 time 322612634496 state 0x2 ref 0 00002000:00100000:3.0:1789762799.311344:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0xf:0x0] (1048576->2097151) req@ffff981ebad83100 x1876702014968960/t4294967794(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.311351:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad83100 with x1876702014968960 ext(1048576->2097151) 00002000:00010000:3.0:1789762799.311354:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad83100 00000100:00100000:0.0:1789762799.315262:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762799.315338:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014969088 00000100:00100000:2.0:1789762799.315346:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014969088 00002000:00100000:2.0:1789762799.315386:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0xf:0x0] (2097152->3145727) req@ffff981d82f93b80 x1876702014969088/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762799.315394:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d82f93b80 with x1876702014969088 ext(2097152->3145727) 00002000:00010000:2.0:1789762799.315398:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d82f93b80 00000100:00100000:2.0:1789762799.315407:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 322656081260 time 322612634496 state 0x0 ref 1 00000100:00100000:2.0:1789762799.315416:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 322662634496 time 322612634496 state 0x1 ref 1 00000100:00100000:2.0:1789762799.315440:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322662634496 time 322619349372 state 0x0 ref 1 00000100:00100000:2.0:1789762799.315461:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014969088:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762799.318014:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702014969088:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2554us (2750us total) trans 4294967795 rc 0/0 00000100:00100000:2.0:1789762799.318026:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 463 00000100:00100000:2.0:1789762799.318036:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322662634496 time 322619349372 state 0x2 ref 0 00002000:00100000:2.0:1789762799.318049:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0xf:0x0] (2097152->3145727) req@ffff981d82f93b80 x1876702014969088/t4294967795(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762799.318060:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d82f93b80 with x1876702014969088 ext(2097152->3145727) 00002000:00010000:2.0:1789762799.318065:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d82f93b80 00000100:00100000:1.0:1789762799.322225:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.322253:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.322296:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014969344 00000100:00100000:3.0:1789762799.322304:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014969344 00000100:00100000:2.0:1789762799.322312:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014969216 00000100:00100000:2.0:1789762799.322319:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014969216 00000100:00100000:3.0:1789762799.322340:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 152 00002000:00100000:2.0:1789762799.322347:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xf:0x0] (3145728->4194303) req@ffff981ebad80e00 x1876702014969216/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00000100:00100000:3.0:1789762799.322349:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:0+-99:2362:x1876702014969344:12345-192.168.203.48@tcp:8:dd.500 00002000:00100000:2.0:1789762799.322387:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad80e00 with x1876702014969216 ext(3145728->4194303) 00002000:00010000:2.0:1789762799.322392:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad80e00 00010000:00080000:3.0:1789762799.322401:0:6647:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp t0 exp 0000000000000000 cur 322 last 0 00000100:00100000:2.0:1789762799.322417:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322662634496 time 322619349372 state 0x0 ref 1 00000100:00100000:2.0:1789762799.322429:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322669349372 time 322619349372 state 0x1 ref 1 00000020:00000080:3.0:1789762799.322441:0:6647:0:(genops.c:1385:class_connect()) connect: client f1bd28de-ee78-4d18-bad3-fa7cec968893, cookie 0x416193e012b03a3f 00002000:00100000:3.0:1789762799.322453:0:6647:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli f1bd28de-ee78-4d18-bad3-fa7cec968893/ffff981d8a640000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:3.0:1789762799.322464:0:6647:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:3.0:1789762799.322632:0:6647:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:3.0:1789762799.322656:0:6647:0:(import.c:65:import_set_state_nolock()) ffff981d8a644000 : changing import state from RECOVER to FULL 00000100:00100000:3.0:1789762799.322706:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2362:x1876702014969344:12345-192.168.203.48@tcp:8:dd.500 Request processed in 359us (452us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762799.322715:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 152 00000100:00100000:1.0:1789762799.323413:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762799.323468:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014969472 00000100:00100000:2.0:1789762799.323475:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014969472 00000100:00080000:1.0:1789762799.323519:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762799.323521:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 153 00000100:00100000:2.0:1789762799.323538:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:13620:x1876702014969472:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:2.0:1789762799.323549:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762799.323571:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005d96e14e/0x416193e012b03a46 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd513 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762799.323598:0:6648:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xa:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789762799.323618:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=322s v=(0 0) 00010000:00010000:2.0:1789762799.323623:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762799.323635:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005d96e14e/0x416193e012b03a46 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd513 expref: 7 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.323656:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005d96e14e/0x416193e012b03a46 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd513 expref: 7 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.323670:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e80031e00, rc 0) 00000100:00100000:2.0:1789762799.323710:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:13620:x1876702014969472:12345-192.168.203.48@tcp:101:dd.500 Request processed in 173us (296us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762799.323719:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 153 00000100:00100000:0.0:1789762799.330596:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.330639:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014969600 00000100:00100000:1.0:1789762799.330644:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014969600 00002000:00100000:1.0:1789762799.330679:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0xa:0x0] (0->1048575) req@ffff981d82f92d80 x1876702014969600/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762799.330692:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d82f92d80 with x1876702014969600 ext(0->1048575) 00002000:00010000:1.0:1789762799.330696:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d82f92d80 00000100:00100000:1.0:1789762799.330710:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322669349372 time 322619349372 state 0x1 ref 2 00000100:00100000:1.0:1789762799.336655:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.336736:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014969728 00000100:00100000:3.0:1789762799.336745:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014969728 00002000:00100000:3.0:1789762799.336781:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xa:0x0] (1048576->2097151) req@ffff981ebad83800 x1876702014969728/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.336807:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad83800 with x1876702014969728 ext(1048576->2097151) 00002000:00010000:3.0:1789762799.336811:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebad83800 00000100:00100000:3.0:1789762799.336827:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322669349372 time 322619349372 state 0x1 ref 3 00000100:00100000:0.0:1789762799.365525:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322719425308 time 322669425308 state 0x1 ref 3 00000100:00100000:0.0:1789762799.365544:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702014969216:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762799.368600:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:2363:x1876702014969216:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3060us (46374us total) trans 4294967796 rc 0/0 00000100:00100000:0.0:1789762799.368609:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 464 00002000:00100000:0.0:1789762799.368618:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xf:0x0] (3145728->4194303) req@ffff981ebad80e00 x1876702014969216/t4294967796(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762799.368625:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad80e00 with x1876702014969216 ext(3145728->4194303) 00002000:00010000:0.0:1789762799.368631:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad80e00 00000100:00100000:2.0:1789762799.415623:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322769512894 time 322719512894 state 0x1 ref 2 00000100:00100000:2.0:1789762799.415651:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702014969600:12345-192.168.203.48@tcp:4:dd.500 00000001:00100000:0.0:1789762799.430683:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000100:00100000:3.0:1789762799.432630:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702014969600:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16983us (102027us total) trans 4294967381 rc 0/0 00000100:00100000:3.0:1789762799.432642:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 465 00002000:00100000:3.0:1789762799.432655:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xa:0x0] (0->1048575) req@ffff981d82f92d80 x1876702014969600/t4294967381(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.432664:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d82f92d80 with x1876702014969600 ext(0->1048575) 00002000:00010000:3.0:1789762799.432670:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d82f92d80 00000100:00100000:2.0:1789762799.435013:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.435069:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014969856 00000100:00100000:1.0:1789762799.435080:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014969856 00002000:00100000:1.0:1789762799.435138:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0xa:0x0] (2097152->3145727) req@ffff981d81394000 x1876702014969856/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762799.435147:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81394000 with x1876702014969856 ext(2097152->3145727) 00002000:00010000:1.0:1789762799.435152:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81394000 00000100:00100000:1.0:1789762799.435171:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322769512894 time 322719512894 state 0x1 ref 2 00000100:00100000:1.0:1789762799.442690:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.442777:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014969984 00000100:00100000:3.0:1789762799.442786:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014969984 00002000:00100000:3.0:1789762799.442817:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xa:0x0] (3145728->4194303) req@ffff981ebad80380 x1876702014969984/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.442826:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad80380 with x1876702014969984 ext(3145728->4194303) 00002000:00010000:3.0:1789762799.442829:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebad80380 00000100:00100000:3.0:1789762799.442844:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322769512894 time 322719512894 state 0x1 ref 3 00000100:00100000:2.0:1789762799.449620:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.449668:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014970112 00000100:00100000:0.0:1789762799.449674:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014970112 00002000:00100000:0.0:1789762799.449700:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xf:0x0] (4194304->5242879) req@ffff981d81395500 x1876702014970112/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762799.449708:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395500 with x1876702014970112 ext(4194304->5242879) 00002000:00010000:0.0:1789762799.449713:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81395500 00000100:00100000:0.0:1789762799.449723:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322769512894 time 322719512894 state 0x1 ref 4 00000100:00100000:2.0:1789762799.457399:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.457464:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014970240 00000100:00100000:1.0:1789762799.457472:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014970240 00002000:00100000:1.0:1789762799.457500:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0xf:0x0] (5242880->6291455) req@ffff981d81396300 x1876702014970240/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762799.457509:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81396300 with x1876702014970240 ext(5242880->6291455) 00002000:00010000:1.0:1789762799.457513:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81396300 00000100:00100000:1.0:1789762799.457524:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322769512894 time 322719512894 state 0x1 ref 5 00000100:00100000:1.0:1789762799.463704:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.463761:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014970368 00000100:00100000:3.0:1789762799.463771:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014970368 00002000:00100000:3.0:1789762799.463806:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xf:0x0] (6291456->7340031) req@ffff981ebad82300 x1876702014970368/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.463835:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad82300 with x1876702014970368 ext(6291456->7340031) 00002000:00010000:3.0:1789762799.463841:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad82300 00000100:00100000:3.0:1789762799.463861:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322769512894 time 322719512894 state 0x1 ref 6 00000100:00100000:0.0:1789762799.465682:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322819584654 time 322769584654 state 0x1 ref 6 00000100:00100000:0.0:1789762799.465699:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014969728:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762799.468497:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702014969728:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2800us (131840us total) trans 4294967382 rc 0/0 00000100:00100000:1.0:1789762799.468506:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 466 00002000:00100000:1.0:1789762799.468515:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xa:0x0] (1048576->2097151) req@ffff981ebad83800 x1876702014969728/t4294967382(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762799.468523:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad83800 with x1876702014969728 ext(1048576->2097151) 00002000:00010000:1.0:1789762799.468528:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebad83800 00000100:00100000:0.0:1789762799.471532:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.471579:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014970496 00000100:00100000:3.0:1789762799.471585:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014970496 00002000:00100000:3.0:1789762799.471610:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xf:0x0] (7340032->8388607) req@ffff981d82f93800 x1876702014970496/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.471619:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d82f93800 with x1876702014970496 ext(7340032->8388607) 00002000:00010000:3.0:1789762799.471622:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d82f93800 00000100:00100000:3.0:1789762799.471634:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322819584654 time 322769584654 state 0x1 ref 6 00000100:00100000:2.0:1789762799.478044:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.478193:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014970624 00000100:00100000:3.0:1789762799.478203:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014970624 00002000:00100000:3.0:1789762799.478241:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xa:0x0] (4194304->5242879) req@ffff981d81397480 x1876702014970624/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.478250:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81397480 with x1876702014970624 ext(4194304->5242879) 00002000:00010000:3.0:1789762799.478254:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81397480 00000100:00100000:3.0:1789762799.478269:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322819584654 time 322769584654 state 0x1 ref 7 00000100:00100000:2.0:1789762799.484176:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.484229:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014970752 00000100:00100000:1.0:1789762799.484237:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014970752 00002000:00100000:1.0:1789762799.484269:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xa:0x0] (5242880->6291455) req@ffff981d81394700 x1876702014970752/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762799.484278:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81394700 with x1876702014970752 ext(5242880->6291455) 00002000:00010000:1.0:1789762799.484282:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81394700 00000100:00100000:1.0:1789762799.484297:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322819584654 time 322769584654 state 0x1 ref 8 00000100:00100000:3.0:1789762799.515845:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322869737569 time 322819737569 state 0x1 ref 8 00000100:00100000:3.0:1789762799.515874:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:2364:x1876702014969856:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762799.519814:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:2364:x1876702014969856:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3949us (84802us total) trans 4294967383 rc 0/0 00000100:00100000:3.0:1789762799.519828:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 467 00002000:00100000:3.0:1789762799.519841:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xa:0x0] (2097152->3145727) req@ffff981d81394000 x1876702014969856/t4294967383(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.519853:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81394000 with x1876702014969856 ext(2097152->3145727) 00002000:00010000:3.0:1789762799.519860:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81394000 00000100:00100000:0.0:1789762799.522267:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.522356:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014970880 00000100:00100000:1.0:1789762799.522385:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014970880 00002000:00100000:1.0:1789762799.522453:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0xa:0x0] (6291456->7340031) req@ffff981d82f92a00 x1876702014970880/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762799.522463:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d82f92a00 with x1876702014970880 ext(6291456->7340031) 00002000:00010000:1.0:1789762799.522467:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d82f92a00 00000100:00100000:1.0:1789762799.522488:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322869737569 time 322819737569 state 0x1 ref 8 00000100:00100000:2.0:1789762799.529022:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762799.529075:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.529080:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014971008 00000100:00100000:0.0:1789762799.529118:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014971008 00000100:00100000:3.0:1789762799.529138:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014971264 00002000:00100000:0.0:1789762799.529151:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xa:0x0] (7340032->8388607) req@ffff981d81395c00 x1876702014971008/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00000100:00100000:3.0:1789762799.529151:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014971264 00002000:00100000:0.0:1789762799.529166:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395c00 with x1876702014971008 ext(7340032->8388607) 00002000:00010000:0.0:1789762799.529170:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81395c00 00000100:00100000:0.0:1789762799.529180:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322869737569 time 322819737569 state 0x1 ref 9 00000100:00100000:3.0:1789762799.529232:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 69 00000100:00100000:3.0:1789762799.529251:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13620:x1876702014971264:12345-192.168.203.48@tcp:35:dd.500 00000100:00100000:3.0:1789762799.529594:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:13620:x1876702014971264:12345-192.168.203.48@tcp:35:dd.500 Request processed in 350us (520us total) trans 4294967489 rc 0/0 00000100:00100000:3.0:1789762799.529604:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 69 00010000:00010000:3.0:1789762799.543281:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762799.564028:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.564071:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014971520 00000100:00100000:3.0:1789762799.564081:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014971520 00000100:00100000:3.0:1789762799.564157:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 238 00000100:00100000:3.0:1789762799.564171:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:13631:x1876702014971520:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762799.564201:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.564226:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a4d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd51a expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.564389:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a54 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.564419:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a54 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.564430:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a54 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.565690:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a5b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.565709:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a5b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.565719:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a5b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762799.565901:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322919801019 time 322869801019 state 0x1 ref 9 00000100:00100000:0.0:1789762799.565918:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:2365:x1876702014969984:12345-192.168.203.48@tcp:4:dd.500 00000004:00080000:3.0:1789762799.566340:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xb:0x0], index 1: 0 00000004:00080000:3.0:1789762799.566352:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x10:0x0], index 0: 0 00010000:00010000:3.0:1789762799.566526:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a62 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.566534:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a62 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.566540:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a62 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.566565:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a54 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.566571:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a54 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.566585:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a54 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.566589:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762799.566592:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a54 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.566598:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a54 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.566607:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:3.0:1789762799.566610:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762799.566613:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a54 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.566618:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03a54 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762799.566628:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b02801dd reply state 000000001809bc4d transno 4294967490 ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a5b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762799.566634:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789762799.566663:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03a4d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd51a expref: 18 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.566674:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a62 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd51a expref: 17 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.566680:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff981e83a46000) 00010000:00010000:3.0:1789762799.568117:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762799.568120:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa33800, rc 0) 00000100:00100000:3.0:1789762799.568243:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:13631:x1876702014971520:12345-192.168.203.48@tcp:101:dd.0 Request processed in 4075us (4221us total) trans 4294967490 rc 0/0 00000100:00100000:3.0:1789762799.568303:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 238 00010000:00010000:0.0:1789762799.569000:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a5b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.569011:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a5b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.569018:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a5b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.569033:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762799.569039:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a5b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762799.569047:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:2365:x1876702014969984:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3129us (126363us total) trans 4294967384 rc 0/0 00010000:00010000:0.0:1789762799.569052:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a5b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762799.569070:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 468 00010000:00010000:0.0:1789762799.569072:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x992ad74f (ffff981e83a46500) 00010000:00010000:0.0:1789762799.569077:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762799.569083:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a5b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00002000:00100000:1.0:1789762799.569084:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xa:0x0] (3145728->4194303) req@ffff981ebad80380 x1876702014969984/t4294967384(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00010000:00010000:0.0:1789762799.569126:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03a5b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00002000:00100000:1.0:1789762799.569129:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad80380 with x1876702014969984 ext(3145728->4194303) 00002000:00010000:1.0:1789762799.569136:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebad80380 00000100:00100000:0.0:1789762799.583434:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.583497:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014971648 00000100:00100000:3.0:1789762799.583506:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014971648 00000100:00100000:3.0:1789762799.583547:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 154 00000100:00100000:3.0:1789762799.583555:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:13631:x1876702014971648:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762799.583569:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.583584:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd521 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762799.583604:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xb:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762799.583618:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=322s v=(0 0) 00010000:00010000:3.0:1789762799.583622:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762799.583629:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 19 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.583640:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 19 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.583648:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa32c00, rc 0) 00000100:00100000:3.0:1789762799.583683:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:13631:x1876702014971648:12345-192.168.203.48@tcp:101:dd.0 Request processed in 128us (256us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762799.583687:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 154 00000100:00100000:2.0:1789762799.585256:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.585300:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014971776 00000100:00100000:3.0:1789762799.585315:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014971776 00000100:00100000:3.0:1789762799.585353:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 155 00000100:00100000:3.0:1789762799.585360:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:13633:x1876702014971776:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762799.585383:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.585395:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000015b64f5/0x416193e012b03a77 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd528 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.585435:0:6647:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 20 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.585447:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762799.585451:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789762799.585458:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000015b64f5/0x416193e012b03a77 lrc: 4/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd528 expref: 20 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.585505:0:6647:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 20 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.585519:0:6647:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 20 pid: 6649 timeout: 422 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762799.585532:0:6647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81b3ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6647:1876702022164096:192.168.203.48@tcp:104: 00000100:00100000:3.0:1789762799.585623:0:6647:0:(client.c:2715:ptlrpc_set_wait()) set ffff981eb450b0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789762799.585842:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762799.585875:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.585897:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014971904 00000100:00100000:2.0:1789762799.585903:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.585904:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014971904 00000100:00100000:0.0:1789762799.585927:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014972032 00000100:00100000:0.0:1789762799.585933:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014972032 00000100:00100000:1.0:1789762799.585940:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014972160 00000100:00100000:1.0:1789762799.585946:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014972160 00000100:00100000:2.0:1789762799.585957:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 156 00000100:00100000:2.0:1789762799.585971:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:13632:x1876702014971904:12345-192.168.203.48@tcp:101:dd.0 00000100:00100000:3.0:1789762799.585971:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 157 00010000:00010000:2.0:1789762799.585983:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1789762799.585988:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:13636:x1876702014972032:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762799.585995:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001bff94e7/0x416193e012b03a7e lrc: 2/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd52f expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.586000:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762799.586016:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xb:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789762799.586023:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(2 0) 00010000:00010000:3.0:1789762799.586025:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007e379827/0x416193e012b03a85 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd536 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.586031:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001bff94e7/0x416193e012b03a7e lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd52f expref: 23 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762799.586046:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x10:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789762799.586049:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001bff94e7/0x416193e012b03a7e lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd52f expref: 23 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.586059:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=322s v=(0 0) 00010000:00010000:2.0:1789762799.586064:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e80030600, rc 0) 00010000:00010000:3.0:1789762799.586064:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762799.586075:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e379827/0x416193e012b03a85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd536 expref: 19 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.586106:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007e379827/0x416193e012b03a85 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd536 expref: 19 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.586115:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa33a00, rc 0) 00000100:00100000:2.0:1789762799.586127:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:13632:x1876702014971904:12345-192.168.203.48@tcp:101:dd.0 Request processed in 159us (281us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762799.586136:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 156 00000100:00100000:3.0:1789762799.586145:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:13636:x1876702014972032:12345-192.168.203.48@tcp:101:dd.0 Request processed in 161us (266us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762799.586154:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 157 00000100:00100000:2.0:1789762799.586164:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 158 00000100:00100000:2.0:1789762799.586175:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:13634:x1876702014972160:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762799.586189:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762799.586196:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000df0d417/0x416193e012b03a8c lrc: 2/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd53d expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.586208:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xb:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762799.586212:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(3 0) 00010000:00010000:2.0:1789762799.586217:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000df0d417/0x416193e012b03a8c lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd53d expref: 23 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.586225:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000df0d417/0x416193e012b03a8c lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd53d expref: 23 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.586232:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e80031a00, rc 0) 00000100:00100000:2.0:1789762799.586261:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:13634:x1876702014972160:12345-192.168.203.48@tcp:101:dd.0 Request processed in 88us (358us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762799.586270:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 158 00000100:00100000:3.0:1789762799.586564:0:6647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81b3ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6647:1876702022164096:192.168.203.48@tcp:104: 00010000:00010000:3.0:1789762799.586629:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000015b64f5/0x416193e012b03a77 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd528 expref: 22 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.586638:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa32800, rc 0) 00000100:00100000:3.0:1789762799.586664:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:13633:x1876702014971776:12345-192.168.203.48@tcp:101:dd.0 Request processed in 1304us (1410us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762799.586673:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 155 00000100:00100000:0.0:1789762799.592954:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.592989:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014972416 00000100:00100000:1.0:1789762799.592996:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014972416 00000100:00100000:1.0:1789762799.593029:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 159 00000100:00100000:1.0:1789762799.593035:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:13642:x1876702014972416:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762799.593044:0:10916:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762799.593052:0:10916:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001f99d421/0x416193e012b03a93 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd54b expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.593068:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xb:0x0].0x0 since=0s v=(3 0) 00010000:00010000:1.0:1789762799.593072:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(4 0) 00010000:00010000:1.0:1789762799.593076:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f99d421/0x416193e012b03a93 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 6 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd54b expref: 23 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.593102:0:10916:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001f99d421/0x416193e012b03a93 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 6 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd54b expref: 23 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.593111:0:10916:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb4c00, rc 0) 00000100:00100000:1.0:1789762799.593155:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:13642:x1876702014972416:12345-192.168.203.48@tcp:101:dd.0 Request processed in 121us (203us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762799.593161:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 159 00000100:00100000:1.0:1789762799.597027:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.597068:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014972288 00000100:00100000:3.0:1789762799.597073:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014972288 00000100:00100000:3.0:1789762799.597119:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 160 00000100:00100000:3.0:1789762799.597126:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:13641:x1876702014972288:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762799.597133:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.597141:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7adbbcd/0x416193e012b03a9a lrc: 2/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd544 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.597155:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xb:0x0].0x0 since=0s v=(4 0) 00010000:00010000:3.0:1789762799.597158:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(5 0) 00010000:00010000:3.0:1789762799.597162:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7adbbcd/0x416193e012b03a9a lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 7 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd544 expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.597170:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7adbbcd/0x416193e012b03a9a lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 7 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd544 expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.597177:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa32400, rc 0) 00000100:00100000:3.0:1789762799.597203:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:13641:x1876702014972288:12345-192.168.203.48@tcp:101:dd.0 Request processed in 78us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762799.597207:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 160 00000100:00100000:2.0:1789762799.598175:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.598214:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014972544 00000100:00100000:3.0:1789762799.598219:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014972544 00000100:00100000:3.0:1789762799.598244:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 161 00000100:00100000:3.0:1789762799.598250:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:13640:x1876702014972544:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762799.598258:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762799.598265:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000008c3282a/0x416193e012b03aa1 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd552 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.598279:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xb:0x0].0x0 since=0s v=(5 0) 00010000:00010000:3.0:1789762799.598288:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:3.0:1789762799.598292:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000008c3282a/0x416193e012b03aa1 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd552 expref: 25 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.598301:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000008c3282a/0x416193e012b03aa1 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd552 expref: 25 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762799.598309:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa33200, rc 0) 00000100:00100000:3.0:1789762799.598336:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:13640:x1876702014972544:12345-192.168.203.48@tcp:101:dd.0 Request processed in 85us (162us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762799.598344:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 161 00000100:00100000:0.0:1789762799.599041:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.599082:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014972672 00000100:00100000:1.0:1789762799.599102:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014972672 00000100:00100000:1.0:1789762799.599127:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 162 00000100:00100000:1.0:1789762799.599133:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:13639:x1876702014972672:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762799.599140:0:10916:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762799.599147:0:10916:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b03aa8 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd559 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.599160:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xb:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762799.599164:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762799.599169:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b03aa8 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd559 expref: 26 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.599176:0:10916:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b03aa8 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd559 expref: 26 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762799.599183:0:10916:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb5000, rc 0) 00000100:00100000:1.0:1789762799.599207:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:13639:x1876702014972672:12345-192.168.203.48@tcp:101:dd.0 Request processed in 74us (168us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762799.599211:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 162 00000100:00100000:3.0:1789762799.615970:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 322969869037 time 322919869037 state 0x1 ref 8 00000100:00100000:3.0:1789762799.615987:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:2366:x1876702014970112:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762799.616156:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.616215:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014972800 00000100:00100000:0.0:1789762799.616220:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014972800 00002000:00100000:0.0:1789762799.616257:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x10:0x0] (5242880->6291455) req@ffff981ebad81c00 x1876702014972800/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762799.616269:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad81c00 with x1876702014972800 ext(5242880->6291455) 00002000:00010000:0.0:1789762799.616276:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad81c00 00000100:00100000:0.0:1789762799.616292:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293269902607 time 293270028593 state 0x0 ref 1 00000100:00100000:0.0:1789762799.616313:0:12581:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 6: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45700 gen 0 rate 20 depth 3 token 0 deadline 293269902607 time 293270028593 state 0x0 ref 1 00000100:00100000:0.0:1789762799.616358:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 3 deadline 322920359950 time 322920259950 state 0x1 ref 1 00000100:00100000:0.0:1789762799.616402:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322920359950 time 322920309269 state 0x0 ref 1 00000100:00100000:0.0:1789762799.616415:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:2363:x1876702014972800:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762799.619572:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.619628:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014972928 00000100:00100000:0.0:1789762799.619634:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014972928 00002000:00100000:0.0:1789762799.619661:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x10:0x0] (6291456->7340031) req@ffff981ebad80700 x1876702014972928/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762799.619672:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad80700 with x1876702014972928 ext(6291456->7340031) 00002000:00010000:0.0:1789762799.619676:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad80700 00000100:00100000:0.0:1789762799.619714:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322920409269 time 322920309269 state 0x1 ref 2 00000100:00100000:1.0:1789762799.619752:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322920409269 time 322923653709 state 0x0 ref 2 00000100:00100000:1.0:1789762799.619777:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:2365:x1876702014972928:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762799.620831:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2366:x1876702014970112:12345-192.168.203.48@tcp:4:dd.500 Request processed in 4847us (171211us total) trans 4294967797 rc 0/0 00000100:00100000:0.0:1789762799.620838:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 469 00002000:00100000:0.0:1789762799.620845:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xf:0x0] (4194304->5242879) req@ffff981d81395500 x1876702014970112/t4294967797(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762799.620852:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395500 with x1876702014970112 ext(4194304->5242879) 00002000:00010000:0.0:1789762799.620857:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81395500 00000100:00100000:0.0:1789762799.627604:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2363:x1876702014972800:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11187us (11445us total) trans 4294967798 rc 0/0 00000100:00100000:2.0:1789762799.627604:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2365:x1876702014972928:12345-192.168.203.48@tcp:4:dd.0 Request processed in 7824us (8025us total) trans 4294967799 rc 0/0 00000100:00100000:0.0:1789762799.627617:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 477 00000100:00100000:2.0:1789762799.627625:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 478 00002000:00100000:0.0:1789762799.627628:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x10:0x0] (5242880->6291455) req@ffff981ebad81c00 x1876702014972800/t4294967798(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762799.627642:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad81c00 with x1876702014972800 ext(5242880->6291455) 00002000:00010000:0.0:1789762799.627650:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad81c00 00000100:00100000:2.0:1789762799.627650:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322920409269 time 322923653709 state 0x2 ref 0 00002000:00100000:2.0:1789762799.627669:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x10:0x0] (6291456->7340031) req@ffff981ebad80700 x1876702014972928/t4294967799(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.627681:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad80700 with x1876702014972928 ext(6291456->7340031) 00002000:00010000:2.0:1789762799.627694:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad80700 00000100:00100000:1.0:1789762799.660277:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762799.660347:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014973056 00000100:00100000:3.0:1789762799.660356:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014973056 00002000:00100000:3.0:1789762799.660421:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x10:0x0] (2097152->3145727) req@ffff981ebad83480 x1876702014973056/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762799.660435:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad83480 with x1876702014973056 ext(2097152->3145727) 00002000:00010000:3.0:1789762799.660441:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad83480 00000100:00100000:3.0:1789762799.660457:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322920409269 time 322923653709 state 0x0 ref 1 00000100:00100000:3.0:1789762799.660489:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322923753709 time 322923653709 state 0x1 ref 1 00000100:00100000:3.0:1789762799.660514:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322923753709 time 322964422364 state 0x0 ref 1 00000100:00100000:3.0:1789762799.660538:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2364:x1876702014973056:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762799.663808:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.663857:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014973184 00000100:00100000:0.0:1789762799.663863:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014973184 00002000:00100000:0.0:1789762799.663890:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x10:0x0] (4194304->5242879) req@ffff981d81b3ad80 x1876702014973184/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762799.663902:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81b3ad80 with x1876702014973184 ext(4194304->5242879) 00002000:00010000:0.0:1789762799.663907:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81b3ad80 00000100:00100000:0.0:1789762799.663945:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322964522364 time 322964422364 state 0x1 ref 2 00000100:00100000:0.0:1789762799.663969:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322964522364 time 322967876784 state 0x0 ref 2 00000100:00100000:0.0:1789762799.663983:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:2363:x1876702014973184:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762799.664149:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.664213:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014973312 00000100:00100000:0.0:1789762799.664219:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014973312 00002000:00100000:0.0:1789762799.664247:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xb:0x0] (0->1048575) req@ffff981d81b3b800 x1876702014973312/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762799.664258:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81b3b800 with x1876702014973312 ext(0->1048575) 00010000:00010000:0.0:1789762799.664269:0:6654:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 26 pid: 6649 timeout: 422 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762799.664299:0:6654:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 26 pid: 6649 timeout: 422 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762799.664309:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981d81b3b800 00000100:00100000:0.0:1789762799.664323:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45a00 gen 0 rate 20 depth 3 token 0 deadline 286879635160 time 286843350654 state 0x0 ref 1 00000100:00100000:0.0:1789762799.664332:0:6654:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 6: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45a00 gen 0 rate 20 depth 3 token 0 deadline 286879635160 time 286843350654 state 0x0 ref 1 00000100:00100000:0.0:1789762799.664382:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 3 deadline 322968373672 time 322968273672 state 0x1 ref 1 00000100:00100000:0.0:1789762799.664412:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 322968373672 time 322968319377 state 0x0 ref 1 00000100:00100000:0.0:1789762799.664425:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:2366:x1876702014973312:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762799.664899:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2364:x1876702014973056:12345-192.168.203.48@tcp:4:dd.0 Request processed in 4363us (4628us total) trans 4294967800 rc 0/0 00000100:00100000:2.0:1789762799.664908:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 479 00002000:00100000:2.0:1789762799.664916:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x10:0x0] (2097152->3145727) req@ffff981ebad83480 x1876702014973056/t4294967800(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.664923:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad83480 with x1876702014973056 ext(2097152->3145727) 00002000:00010000:2.0:1789762799.664927:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad83480 00000100:00100000:1.0:1789762799.666009:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323019913093 time 322969913093 state 0x1 ref 7 00000100:00100000:1.0:1789762799.666023:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81396300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2365:x1876702014970240:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762799.671287:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762799.671315:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.671337:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014973440 00000100:00100000:1.0:1789762799.671344:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014973440 00000100:00100000:3.0:1789762799.671347:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014973568 00000100:00100000:2.0:1789762799.671355:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00002000:00100000:1.0:1789762799.671381:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x10:0x0] (1048576->2097151) req@ffff981e98e7ad80 x1876702014973440/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762799.671390:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7ad80 with x1876702014973440 ext(1048576->2097151) 00002000:00010000:1.0:1789762799.671395:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7ad80 00000100:00100000:0.0:1789762799.671404:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014973696 00000100:00100000:1.0:1789762799.671408:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322967976784 time 322967876784 state 0x1 ref 2 00000100:00100000:0.0:1789762799.671412:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014973696 00000100:00100000:1.0:1789762799.671427:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322967976784 time 322975336115 state 0x0 ref 2 00000100:00100000:3.0:1789762799.671431:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014973568 00000100:00100000:1.0:1789762799.671441:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2364:x1876702014973440:12345-192.168.203.48@tcp:4:dd.0 00002000:00100000:0.0:1789762799.671444:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x10:0x0] (0->1048575) req@ffff981e98e7aa00 x1876702014973696/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762799.671460:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7aa00 with x1876702014973696 ext(0->1048575) 00002000:00100000:3.0:1789762799.671460:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x10:0x0] (7340032->8388607) req@ffff981e98e7b800 x1876702014973568/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00010000:0.0:1789762799.671467:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7aa00 00002000:00100000:3.0:1789762799.671479:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7b800 with x1876702014973568 ext(7340032->8388607) 00000100:00100000:2.0:1789762799.671484:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.671493:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322975436115 time 322975336115 state 0x1 ref 3 00000100:00100000:0.0:1789762799.671510:0:12582:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 6 min 3 max 116 00000100:00100000:0.0:1789762799.671526:0:12582:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_006' 00002000:00010000:3.0:1789762799.671575:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7b800 00000100:00100000:3.0:1789762799.671586:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322975436115 time 322975336115 state 0x1 ref 4 00000100:00100000:3.0:1789762799.671653:0:6652:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 6 min 3 max 116 00000100:00100000:3.0:1789762799.671659:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014973824 00000100:00100000:3.0:1789762799.671662:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014973824 00002000:00100000:3.0:1789762799.671678:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x10:0x0] (3145728->4194303) req@ffff981e98e7a680 x1876702014973824/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762799.671685:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7a680 with x1876702014973824 ext(3145728->4194303) 00002000:00010000:3.0:1789762799.671688:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7a680 00000100:00100000:0.0:1789762799.671741:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322975745359 time 322975645359 state 0x1 ref 4 00000100:00100000:0.0:1789762799.671757:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2365:x1876702014973696:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762799.671866:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322975745359 time 322975645359 state 0x1 ref 5 00000100:00100000:1.0:1789762799.672316:0:14912:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 7 min 3 max 116 00000100:00100000:1.0:1789762799.672322:0:14912:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_007' 00000100:00100000:0.0:1789762799.672477:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322976481075 time 322976381075 state 0x1 ref 5 00000100:00100000:0.0:1789762799.672495:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2363:x1876702014973568:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762799.672956:0:14913:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 8 min 3 max 116 00000100:00100000:1.0:1789762799.672964:0:14913:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_008' 00000100:00100000:1.0:1789762799.673043:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322976481075 time 322976950346 state 0x0 ref 5 00000100:00100000:1.0:1789762799.673054:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2363:x1876702014973824:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762799.673338:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2363:x1876702014973184:12345-192.168.203.48@tcp:4:dd.0 Request processed in 9357us (9530us total) trans 4294967801 rc 0/0 00000100:00100000:2.0:1789762799.673347:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 480 00002000:00100000:2.0:1789762799.673354:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x10:0x0] (4194304->5242879) req@ffff981d81b3ad80 x1876702014973184/t4294967801(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.673374:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81b3ad80 with x1876702014973184 ext(4194304->5242879) 00002000:00010000:2.0:1789762799.673379:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81b3ad80 00000100:00100000:2.0:1789762799.673394:0:6653:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 8 min 3 max 116 00000100:00100000:2.0:1789762799.677386:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2366:x1876702014973312:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12958us (13234us total) trans 4294967385 rc 0/0 00000100:00100000:3.0:1789762799.677386:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81396300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2365:x1876702014970240:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11344us (219969us total) trans 4294967802 rc 0/0 00000100:00100000:2.0:1789762799.677401:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 5 00000100:00100000:2.0:1789762799.677414:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 322968373672 time 322968319377 state 0x2 ref 0 00002000:00100000:2.0:1789762799.677442:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xb:0x0] (0->1048575) req@ffff981d81b3b800 x1876702014973312/t4294967385(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.677450:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81b3b800 with x1876702014973312 ext(0->1048575) 00010000:00010000:2.0:1789762799.677459:0:6654:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 28 pid: 6649 timeout: 422 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.677474:0:6654:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 28 pid: 6649 timeout: 422 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762799.677481:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981d81b3b800 00000100:00100000:3.0:1789762799.677763:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 470 00002000:00100000:3.0:1789762799.677773:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0xf:0x0] (5242880->6291455) req@ffff981d81396300 x1876702014970240/t4294967802(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.677848:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81396300 with x1876702014970240 ext(5242880->6291455) 00002000:00010000:3.0:1789762799.677853:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81396300 00000100:00100000:2.0:1789762799.677971:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2364:x1876702014973440:12345-192.168.203.48@tcp:4:dd.0 Request processed in 6531us (6682us total) trans 4294967803 rc 0/0 00000100:00100000:2.0:1789762799.677983:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 481 00002000:00100000:2.0:1789762799.677992:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x10:0x0] (1048576->2097151) req@ffff981e98e7ad80 x1876702014973440/t4294967803(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.678002:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7ad80 with x1876702014973440 ext(1048576->2097151) 00002000:00010000:2.0:1789762799.678008:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7ad80 00000100:00100000:2.0:1789762799.679179:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2365:x1876702014973696:12345-192.168.203.48@tcp:4:dd.0 Request processed in 7424us (7821us total) trans 4294967804 rc 0/0 00000100:00100000:2.0:1789762799.679188:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 482 00002000:00100000:2.0:1789762799.679207:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x10:0x0] (0->1048575) req@ffff981e98e7aa00 x1876702014973696/t4294967804(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.679218:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7aa00 with x1876702014973696 ext(0->1048575) 00002000:00010000:2.0:1789762799.679228:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7aa00 00000100:00100000:0.0:1789762799.680075:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762799.680128:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014973952 00000100:00100000:2.0:1789762799.680136:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014973952 00010000:00010000:2.0:1789762799.680165:0:5790:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 28 pid: 6649 timeout: 422 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762799.680197:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 59 00000100:00100000:2.0:1789762799.680209:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2363:x1876702014973952:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762799.680224:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1789762799.680239:0:5790:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xb:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:2.0:1789762799.680242:0:5790:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xb:0x0] updating lvb mtime from disk: 0 -> 1789762799 00002000:00010000:2.0:1789762799.680244:0:5790:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xb:0x0] updating lvb ctime from disk: 0 -> 1789762799 00010000:00010000:2.0:1789762799.680250:0:5790:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 28 pid: 6649 timeout: 422 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680263:0:5790:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 28 pid: 6649 timeout: 422 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680289:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c25d8979/0x416193e012b03a70 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd521 expref: 28 pid: 6649 timeout: 422 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680305:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680310:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680320:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000015b64f5/0x416193e012b03a77 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd528 expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680329:0:5790:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000015b64f5/0x416193e012b03a77 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd528 expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680345:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680347:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680352:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001bff94e7/0x416193e012b03a7e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd52f expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680358:0:5790:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001bff94e7/0x416193e012b03a7e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd52f expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680376:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680377:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680382:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000df0d417/0x416193e012b03a8c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd53d expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680388:0:5790:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000df0d417/0x416193e012b03a8c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd53d expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680393:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680395:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680398:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001f99d421/0x416193e012b03a93 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd54b expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680403:0:5790:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001f99d421/0x416193e012b03a93 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd54b expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680407:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680414:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680419:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7adbbcd/0x416193e012b03a9a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd544 expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680426:0:5790:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7adbbcd/0x416193e012b03a9a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd544 expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680432:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680434:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680437:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000008c3282a/0x416193e012b03aa1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd552 expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680444:0:5790:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000008c3282a/0x416193e012b03aa1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd552 expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680448:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680450:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762799.680454:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b03aa8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd559 expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680460:0:5790:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b03aa8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd559 expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762799.680498:0:5790:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b03aa8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd559 expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762799.680513:0:5790:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e79180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164224:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762799.680630:0:5790:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000008c3282a/0x416193e012b03aa1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd552 expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762799.680645:0:5790:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164352:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762799.680698:0:5790:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e7adbbcd/0x416193e012b03a9a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd544 expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762799.680712:0:5790:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e78e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164480:192.168.203.48@tcp:105: 00000100:00100000:0.0:1789762799.680735:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2363:x1876702014973568:12345-192.168.203.48@tcp:4:dd.0 Request processed in 8240us (9414us total) trans 4294967805 rc 0/0 00000100:00100000:0.0:1789762799.680748:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 483 00010000:00010000:2.0:1789762799.680762:0:5790:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001f99d421/0x416193e012b03a93 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd54b expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:0.0:1789762799.680763:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x240000400:0x10:0x0] (7340032->8388607) req@ffff981e98e7b800 x1876702014973568/t4294967805(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762799.680792:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7b800 with x1876702014973568 ext(7340032->8388607) 00000100:00100000:2.0:1789762799.680796:0:5790:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e78a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164608:192.168.203.48@tcp:105: 00002000:00010000:0.0:1789762799.680800:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7b800 00010000:00010000:2.0:1789762799.680834:0:5790:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000000df0d417/0x416193e012b03a8c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd53d expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762799.680847:0:5790:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e78000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164736:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762799.680877:0:5790:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000001bff94e7/0x416193e012b03a7e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd52f expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762799.680889:0:5790:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e79500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164864:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762799.680923:0:5790:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000015b64f5/0x416193e012b03a77 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd528 expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762799.680950:0:5790:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164992:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762799.680977:0:5790:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebf4c3a80 going to sleep for 11 seconds 00000100:00100000:0.0:1789762799.681650:0:5790:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e79180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164224:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762799.681719:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2363:x1876702014973824:12345-192.168.203.48@tcp:4:dd.0 Request processed in 8664us (10236us total) trans 4294967806 rc 0/0 00000100:00100000:2.0:1789762799.681727:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 484 00000100:00100000:2.0:1789762799.681735:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322976481075 time 322976950346 state 0x2 ref 0 00002000:00100000:2.0:1789762799.681744:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x10:0x0] (3145728->4194303) req@ffff981e98e7a680 x1876702014973824/t4294967806(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.681755:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7a680 with x1876702014973824 ext(3145728->4194303) 00002000:00010000:2.0:1789762799.681759:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7a680 00000100:00100000:3.0:1789762799.681797:0:5790:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7b100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164352:192.168.203.48@tcp:105: 00000100:00100000:3.0:1789762799.691138:0:5790:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e78e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164480:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762799.691207:0:5790:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e78a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164608:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762799.691217:0:5790:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e78000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164736:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762799.695874:0:5790:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e79500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164864:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762799.695886:0:5790:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7a300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5790:1876702022164992:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762799.695912:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762799.695954:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2363:x1876702014973952:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 15746us (15877us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762799.695965:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 59 00000100:00100000:3.0:1789762799.716117:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323069996034 time 323019996034 state 0x1 ref 6 00000100:00100000:3.0:1789762799.716145:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2365:x1876702014970368:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762799.718147:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762799.718197:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014974080 00000100:00100000:2.0:1789762799.718206:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014974080 00002000:00100000:2.0:1789762799.718251:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xb:0x0] (5242880->6291455) req@ffff981ebad81880 x1876702014974080/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.718272:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad81880 with x1876702014974080 ext(5242880->6291455) 00002000:00010000:2.0:1789762799.718279:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebad81880 00000100:00100000:2.0:1789762799.718297:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322976481075 time 322976950346 state 0x0 ref 1 00000100:00100000:2.0:1789762799.718315:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322977050346 time 322976950346 state 0x1 ref 1 00000100:00100000:2.0:1789762799.718346:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 322977050346 time 323022252221 state 0x0 ref 1 00000100:00100000:2.0:1789762799.718373:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2364:x1876702014974080:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762799.718794:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762799.718843:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014974208 00000100:00100000:2.0:1789762799.718849:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014974208 00002000:00100000:2.0:1789762799.718873:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xb:0x0] (4194304->5242879) req@ffff981d82f90000 x1876702014974208/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.718884:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d82f90000 with x1876702014974208 ext(4194304->5242879) 00002000:00010000:2.0:1789762799.718889:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d82f90000 00000100:00100000:2.0:1789762799.718919:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323022352221 time 323022252221 state 0x1 ref 2 00000100:00100000:2.0:1789762799.718940:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323022352221 time 323022847818 state 0x0 ref 2 00000100:00100000:2.0:1789762799.718954:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2363:x1876702014974208:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762799.724212:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702014970368:12345-192.168.203.48@tcp:4:dd.500 Request processed in 8077us (260506us total) trans 4294967807 rc 0/0 00000100:00100000:2.0:1789762799.724232:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 471 00002000:00100000:2.0:1789762799.724240:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xf:0x0] (6291456->7340031) req@ffff981ebad82300 x1876702014970368/t4294967807(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762799.724248:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad82300 with x1876702014970368 ext(6291456->7340031) 00002000:00010000:2.0:1789762799.724253:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad82300 00000100:00100000:2.0:1789762799.726199:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2364:x1876702014974080:12345-192.168.203.48@tcp:4:dd.0 Request processed in 7840us (8051us total) trans 4294967386 rc 0/0 00000100:00100000:2.0:1789762799.726206:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 485 00002000:00100000:2.0:1789762799.726213:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xb:0x0] (5242880->6291455) req@ffff981ebad81880 x1876702014974080/t4294967386(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.726221:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad81880 with x1876702014974080 ext(5242880->6291455) 00002000:00010000:2.0:1789762799.726225:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebad81880 00000100:00100000:2.0:1789762799.727505:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2363:x1876702014974208:12345-192.168.203.48@tcp:4:dd.0 Request processed in 8552us (8711us total) trans 4294967387 rc 0/0 00000100:00100000:2.0:1789762799.727512:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 486 00000100:00100000:2.0:1789762799.727520:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323022352221 time 323022847818 state 0x2 ref 0 00002000:00100000:2.0:1789762799.727530:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xb:0x0] (4194304->5242879) req@ffff981d82f90000 x1876702014974208/t4294967387(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.727537:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d82f90000 with x1876702014974208 ext(4194304->5242879) 00002000:00010000:2.0:1789762799.727541:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d82f90000 00000100:00100000:1.0:1789762799.736539:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762799.736605:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014974336 00000100:00100000:2.0:1789762799.736616:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014974336 00002000:00100000:2.0:1789762799.736687:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xb:0x0] (3145728->4194303) req@ffff981ebad81500 x1876702014974336/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.736702:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad81500 with x1876702014974336 ext(3145728->4194303) 00002000:00010000:2.0:1789762799.736709:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebad81500 00000100:00100000:2.0:1789762799.736728:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323022352221 time 323022847818 state 0x0 ref 1 00000100:00100000:2.0:1789762799.736781:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323022947818 time 323022847818 state 0x1 ref 1 00000100:00100000:2.0:1789762799.736817:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323022947818 time 323040724431 state 0x0 ref 1 00000100:00100000:2.0:1789762799.736835:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2364:x1876702014974336:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762799.738114:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762799.738161:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014974464 00000100:00100000:2.0:1789762799.738167:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014974464 00000100:00100000:1.0:1789762799.738174:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00002000:00100000:2.0:1789762799.738195:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0xb:0x0] (2097152->3145727) req@ffff981ebfecd500 x1876702014974464/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.738203:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfecd500 with x1876702014974464 ext(2097152->3145727) 00002000:00010000:2.0:1789762799.738208:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfecd500 00000100:00100000:3.0:1789762799.738221:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014974592 00000100:00100000:3.0:1789762799.738229:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014974592 00000100:00100000:2.0:1789762799.738241:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323040824431 time 323040724431 state 0x1 ref 2 00002000:00100000:3.0:1789762799.738264:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xb:0x0] (7340032->8388607) req@ffff981d8aa79f80 x1876702014974592/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1789762799.738288:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323040824431 time 323042179975 state 0x0 ref 2 00002000:00100000:3.0:1789762799.738289:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d8aa79f80 with x1876702014974592 ext(7340032->8388607) 00002000:00010000:3.0:1789762799.738298:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d8aa79f80 00000100:00100000:1.0:1789762799.738311:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfecd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2364:x1876702014974464:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762799.738329:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323042279975 time 323042179975 state 0x1 ref 3 00000100:00100000:2.0:1789762799.738398:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323042279975 time 323042304898 state 0x0 ref 3 00000100:00100000:2.0:1789762799.738410:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2364:x1876702014974592:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762799.745217:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2364:x1876702014974592:12345-192.168.203.48@tcp:4:dd.0 Request processed in 6801us (7037us total) trans 4294967388 rc 0/0 00000100:00100000:2.0:1789762799.745240:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 489 00002000:00100000:2.0:1789762799.745255:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xb:0x0] (7340032->8388607) req@ffff981d8aa79f80 x1876702014974592/t4294967388(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.745269:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d8aa79f80 with x1876702014974592 ext(7340032->8388607) 00002000:00010000:2.0:1789762799.745276:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d8aa79f80 00000100:00100000:1.0:1789762799.745596:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfecd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2364:x1876702014974464:12345-192.168.203.48@tcp:4:dd.0 Request processed in 7290us (7482us total) trans 4294967389 rc 0/0 00000100:00100000:1.0:1789762799.745603:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 488 00002000:00100000:1.0:1789762799.745610:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0xb:0x0] (2097152->3145727) req@ffff981ebfecd500 x1876702014974464/t4294967389(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762799.745617:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfecd500 with x1876702014974464 ext(2097152->3145727) 00002000:00010000:1.0:1789762799.745620:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfecd500 00000100:00100000:1.0:1789762799.746313:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762799.746336:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.746372:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014974976 00000100:00100000:1.0:1789762799.746372:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762799.746382:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014974976 00000100:00100000:2.0:1789762799.746402:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014975104 00000100:00100000:2.0:1789762799.746408:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014975104 00000100:00100000:1.0:1789762799.746422:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014974720 00000100:00100000:1.0:1789762799.746430:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014974720 00002000:00100000:2.0:1789762799.746435:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0xb:0x0] (6291456->7340031) req@ffff981d8aa79880 x1876702014975104/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1789762799.746440:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 70 00002000:00100000:2.0:1789762799.746450:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d8aa79880 with x1876702014975104 ext(6291456->7340031) 00002000:00010000:2.0:1789762799.746454:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d8aa79880 00002000:00100000:1.0:1789762799.746458:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0xb:0x0] (1048576->2097151) req@ffff981d8aa78380 x1876702014974720/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/0 e 0 to 0 dl 1789762810 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00000100:00100000:0.0:1789762799.746460:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b1b850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:13632:x1876702014974976:12345-192.168.203.48@tcp:35:dd.0 00002000:00100000:1.0:1789762799.746473:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d8aa78380 with x1876702014974720 ext(1048576->2097151) 00002000:00010000:1.0:1789762799.746478:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d8aa78380 00000100:00100000:2.0:1789762799.746492:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323042404898 time 323042304898 state 0x1 ref 3 00000100:00100000:1.0:1789762799.746506:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323042404898 time 323042304898 state 0x1 ref 3 00000100:00100000:2.0:1789762799.746526:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323050532720 time 323050432720 state 0x1 ref 3 00000100:00100000:2.0:1789762799.746538:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702014975104:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762799.746548:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 323050532720 time 323050446761 state 0x0 ref 3 00000100:00100000:1.0:1789762799.746566:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2366:x1876702014974720:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762799.746782:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b1b850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:13632:x1876702014974976:12345-192.168.203.48@tcp:35:dd.0 Request processed in 328us (467us total) trans 4294967491 rc 0/0 00000100:00100000:0.0:1789762799.746804:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 70 00000100:00100000:1.0:1789762799.747190:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2364:x1876702014974336:12345-192.168.203.48@tcp:4:dd.0 Request processed in 10356us (10661us total) trans 4294967390 rc 0/0 00000100:00100000:1.0:1789762799.747201:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 487 00002000:00100000:1.0:1789762799.747210:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xb:0x0] (3145728->4194303) req@ffff981ebad81500 x1876702014974336/t4294967390(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762799.747221:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad81500 with x1876702014974336 ext(3145728->4194303) 00002000:00010000:1.0:1789762799.747227:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebad81500 00000100:00100000:2.0:1789762799.749564:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa79880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2365:x1876702014975104:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3024us (3220us total) trans 4294967391 rc 0/0 00000100:00100000:2.0:1789762799.749578:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 490 00002000:00100000:2.0:1789762799.749588:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0xb:0x0] (6291456->7340031) req@ffff981d8aa79880 x1876702014975104/t4294967391(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.749599:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d8aa79880 with x1876702014975104 ext(6291456->7340031) 00002000:00010000:2.0:1789762799.749605:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d8aa79880 00000100:00100000:2.0:1789762799.750563:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2366:x1876702014974720:12345-192.168.203.48@tcp:4:dd.0 Request processed in 4002us (4201us total) trans 4294967392 rc 0/0 00000100:00100000:2.0:1789762799.750570:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 491 00000100:00100000:2.0:1789762799.750577:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 323050532720 time 323050446761 state 0x2 ref 0 00002000:00100000:2.0:1789762799.750586:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0xb:0x0] (1048576->2097151) req@ffff981d8aa78380 x1876702014974720/t4294967392(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762799.750593:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d8aa78380 with x1876702014974720 ext(1048576->2097151) 00002000:00010000:2.0:1789762799.750597:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d8aa78380 00000100:00100000:1.0:1789762799.766169:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323120067691 time 323070067691 state 0x1 ref 5 00000100:00100000:1.0:1789762799.766190:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2366:x1876702014970496:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762799.768941:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2366:x1876702014970496:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2753us (297404us total) trans 4294967808 rc 0/0 00000100:00100000:3.0:1789762799.768953:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 472 00002000:00100000:3.0:1789762799.768967:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xf:0x0] (7340032->8388607) req@ffff981d82f93800 x1876702014970496/t4294967808(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.768980:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d82f93800 with x1876702014970496 ext(7340032->8388607) 00002000:00010000:3.0:1789762799.768988:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d82f93800 00000100:00100000:2.0:1789762799.816274:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323170164002 time 323120164002 state 0x1 ref 4 00000100:00100000:2.0:1789762799.816302:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81397480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2366:x1876702014970624:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762799.819561:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81397480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2366:x1876702014970624:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3267us (341519us total) trans 4294967393 rc 0/0 00000100:00100000:2.0:1789762799.819569:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 473 00002000:00100000:2.0:1789762799.819581:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xa:0x0] (4194304->5242879) req@ffff981d81397480 x1876702014970624/t4294967393(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762799.819590:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81397480 with x1876702014970624 ext(4194304->5242879) 00002000:00010000:2.0:1789762799.819597:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81397480 00000100:00100000:1.0:1789762799.866373:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323220250513 time 323170250513 state 0x1 ref 3 00000100:00100000:1.0:1789762799.866399:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2365:x1876702014970752:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762799.869329:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:2365:x1876702014970752:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2934us (385154us total) trans 4294967394 rc 0/0 00000100:00100000:3.0:1789762799.869338:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 474 00002000:00100000:3.0:1789762799.869348:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0xa:0x0] (5242880->6291455) req@ffff981d81394700 x1876702014970752/t4294967394(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.869355:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81394700 with x1876702014970752 ext(5242880->6291455) 00002000:00010000:3.0:1789762799.869375:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81394700 00000100:00100000:2.0:1789762799.916453:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323270343035 time 323220343035 state 0x1 ref 2 00000100:00100000:2.0:1789762799.916478:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2365:x1876702014970880:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762799.919589:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:2365:x1876702014970880:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3117us (397326us total) trans 4294967395 rc 0/0 00000100:00100000:2.0:1789762799.919606:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 475 00002000:00100000:2.0:1789762799.919617:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xa:0x0] (6291456->7340031) req@ffff981d82f92a00 x1876702014970880/t4294967395(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762799.919626:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d82f92a00 with x1876702014970880 ext(6291456->7340031) 00002000:00010000:2.0:1789762799.919635:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d82f92a00 00000100:00100000:2.0:1789762799.966507:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323270343035 time 323270400528 state 0x0 ref 1 00000100:00100000:2.0:1789762799.966530:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:2366:x1876702014971008:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762799.973393:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+45:2366:x1876702014971008:12345-192.168.203.48@tcp:4:dd.500 Request processed in 6866us (444369us total) trans 4294967396 rc 0/0 00000100:00100000:3.0:1789762799.973402:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 476 00000100:00100000:3.0:1789762799.973412:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323270343035 time 323270400528 state 0x2 ref 0 00002000:00100000:3.0:1789762799.973424:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0xa:0x0] (7340032->8388607) req@ffff981d81395c00 x1876702014971008/t4294967396(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:580/0 lens 488/448 e 0 to 0 dl 1789762810 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762799.973433:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395c00 with x1876702014971008 ext(7340032->8388607) 00002000:00010000:3.0:1789762799.973440:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81395c00 00000100:00100000:2.0:1789762800.016121:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762800.016160:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762800.016206:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014975232 00000100:00100000:3.0:1789762800.016209:0:7595:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014975360 00000100:00100000:3.0:1789762800.016221:0:7595:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014975360 00000100:00100000:0.0:1789762800.016222:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014975232 00000100:00100000:0.0:1789762800.016304:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 17 00000100:00100000:3.0:1789762800.016316:0:7595:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 18 00000100:00100000:0.0:1789762800.016322:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:5592:x1876702014975232:12345-192.168.203.48@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1789762800.016337:0:7595:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e79500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+45:5593:x1876702014975360:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762800.016342:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1789762800.016348:0:7595:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 8 locks, starting at 0 00002000:00010000:0.0:1789762800.016381:0:5791:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:3.0:1789762800.016381:0:7595:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xa:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:3.0:1789762800.016389:0:7595:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xa:0x0] updating lvb mtime from disk: 0 -> 1789762799 00002000:00010000:0.0:1789762800.016390:0:5791:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb mtime from disk: 0 -> 1789762799 00002000:00010000:0.0:1789762800.016397:0:5791:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb ctime from disk: 0 -> 1789762799 00002000:00010000:3.0:1789762800.016397:0:7595:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xa:0x0] updating lvb ctime from disk: 0 -> 1789762799 00010000:00010000:0.0:1789762800.016437:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000eb30978/0x416193e012b03a38 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd50c expref: 39 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.016437:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005d96e14e/0x416193e012b03a46 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd513 expref: 45 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762800.016478:0:5791:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:3.0:1789762800.016487:0:7595:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xb:0x0] updating lvb size from disk: 1048576 -> 8388608 00002000:00010000:0.0:1789762800.016488:0:5791:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb mtime from disk: 0 -> 1789762799 00002000:00010000:0.0:1789762800.016496:0:5791:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb ctime from disk: 0 -> 1789762799 00010000:00010000:3.0:1789762800.016510:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7adbbcd/0x416193e012b03a9a lrc: 0/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd544 expref: 44 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.016511:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007e379827/0x416193e012b03a85 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd536 expref: 38 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.016541:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762800.016551:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001bff94e7/0x416193e012b03a7e lrc: 0/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 8 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd52f expref: 43 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.016564:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001f99d421/0x416193e012b03a93 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 7 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd54b expref: 42 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.016575:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000015b64f5/0x416193e012b03a77 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd528 expref: 41 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.016584:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000df0d417/0x416193e012b03a8c lrc: 0/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd53d expref: 40 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.016592:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000033fe766/0x416193e012b03aa8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 4 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd559 expref: 39 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.016601:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000008c3282a/0x416193e012b03aa1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd552 expref: 38 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762800.016609:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:5592:x1876702014975232:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 294us (493us total) trans 0 rc 0/0 00010000:00010000:3.0:1789762800.016613:0:7595:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762800.016618:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 17 00000100:00100000:3.0:1789762800.016644:0:7595:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e79500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:5593:x1876702014975360:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 316us (482us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762800.016656:0:7595:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 18 00000100:00100000:0.0:1789762800.056144:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.056212:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014975616 00000100:00100000:3.0:1789762800.056229:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014975616 00000100:00100000:3.0:1789762800.056301:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 239 00000100:00100000:3.0:1789762800.056319:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:13667:x1876702014975616:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762800.056353:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762800.056394:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03aaf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd560 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.056513:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03ab6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.056528:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03ab6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.056541:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03ab6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762800.056551:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x27:0x0]: returning size 16777216 00010000:00010000:3.0:1789762800.056695:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03aaf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd560 expref: 21 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.056712:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03ab6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd560 expref: 20 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.056721:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff981e83a46200) 00010000:00010000:3.0:1789762800.056723:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762800.056726:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa32800, rc 0) 00000100:00100000:3.0:1789762800.056811:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:13667:x1876702014975616:12345-192.168.203.48@tcp:101:dd.0 Request processed in 499us (687us total) trans 4294967492 rc 0/0 00000100:00100000:3.0:1789762800.056818:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 239 00000100:00100000:1.0:1789762800.057609:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762800.057634:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762800.057682:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014975872 00000100:00100000:0.0:1789762800.057689:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014975872 00000100:00100000:3.0:1789762800.057693:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014976128 00000100:00100000:3.0:1789762800.057699:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014976128 00000100:00100000:0.0:1789762800.057738:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 240 00000100:00100000:3.0:1789762800.057754:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 241 00000100:00100000:0.0:1789762800.057760:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:13668:x1876702014975872:12345-192.168.203.48@tcp:101:dd.0 00000100:00100000:3.0:1789762800.057769:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:13669:x1876702014976128:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762800.057771:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762800.057781:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762800.057784:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03ac4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd567 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.057792:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32400/0x416193e012b03acb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd56e expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.057829:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03ad2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.057839:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03ad2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.057845:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03ad2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762800.057852:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x27:0x0]: returning size 16777216 00010000:00010000:3.0:1789762800.057862:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03ad9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.057877:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03ad9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.057887:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03ad9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762800.057898:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x27:0x0]: returning size 16777216 00010000:00010000:0.0:1789762800.057926:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03ac4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd567 expref: 25 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.057946:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03ad2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd567 expref: 25 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.057955:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff981e83a46200) 00010000:00010000:0.0:1789762800.057959:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762800.057962:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aa600, rc 0) 00000100:00100000:0.0:1789762800.058001:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:13668:x1876702014975872:12345-192.168.203.48@tcp:101:dd.0 Request processed in 246us (390us total) trans 4294967493 rc 0/0 00000100:00100000:0.0:1789762800.058010:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 240 00000100:00100000:1.0:1789762800.058046:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:3.0:1789762800.058054:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32400/0x416193e012b03acb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd56e expref: 25 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.058071:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03ad9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd56e expref: 24 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.058081:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff981e83a46200) 00000100:00100000:0.0:1789762800.058110:0:8405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014976256 00000100:00100000:0.0:1789762800.058117:0:8405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014976256 00010000:00010000:3.0:1789762800.058156:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762800.058169:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa32e00, rc 0) 00000100:00100000:0.0:1789762800.058183:0:8405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 19 00000100:00100000:0.0:1789762800.058196:0:8405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2363:x1876702014976256:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762800.058210:0:8405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1789762800.058211:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:13669:x1876702014976128:12345-192.168.203.48@tcp:101:dd.0 Request processed in 443us (572us total) trans 4294967494 rc 0/0 00000100:00100000:3.0:1789762800.058225:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 241 00010000:00010000:0.0:1789762800.058232:0:8405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03ab6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd560 expref: 25 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.058245:0:8405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff981e83a46200) 00010000:00010000:0.0:1789762800.058247:0:8405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762800.058268:0:8405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2363:x1876702014976256:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 78us (223us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762800.058274:0:8405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 19 00000100:00100000:2.0:1789762800.058451:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.058575:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014976384 00000100:00100000:3.0:1789762800.058580:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014976384 00000100:00100000:3.0:1789762800.058611:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 163 00000100:00100000:3.0:1789762800.058618:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:13667:x1876702014976384:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762800.058627:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762800.058639:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03aee lrc: 2/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd575 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762800.058660:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xf:0x0] initial LVB size: 8388608, mtime: 0x6aad9cef, atime: 0x0, ctime: 0x6aad9cef, blocks: 0x1 00010000:00010000:3.0:1789762800.058678:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=323s v=(0 0) 00010000:00010000:3.0:1789762800.058685:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.058698:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03aee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd575 expref: 38 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.058713:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03aee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd575 expref: 38 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.058726:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa33c00, rc 0) 00000100:00100000:3.0:1789762800.058757:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:13667:x1876702014976384:12345-192.168.203.48@tcp:101:dd.0 Request processed in 140us (309us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762800.058763:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 163 00000100:00100000:1.0:1789762800.059876:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.059903:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014976640 00000100:00100000:1.0:1789762800.059904:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.059909:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014976640 00000100:00100000:3.0:1789762800.059965:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 71 00000100:00100000:2.0:1789762800.059966:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014976768 00000100:00100000:2.0:1789762800.059971:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014976768 00000100:00100000:3.0:1789762800.059975:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:13669:x1876702014976640:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:2.0:1789762800.060012:0:6664:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 20 00000100:00100000:2.0:1789762800.060021:0:6664:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2366:x1876702014976768:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762800.060029:0:6664:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762800.060040:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03ad9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd56e expref: 24 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.060051:0:6664:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff981e83a46200) 00010000:00010000:2.0:1789762800.060054:0:6664:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762800.060084:0:6664:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2366:x1876702014976768:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 64us (178us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762800.060116:0:6664:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 20 00000100:00100000:3.0:1789762800.060136:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:13669:x1876702014976640:12345-192.168.203.48@tcp:35:dd.0 Request processed in 161us (259us total) trans 4294967495 rc 0/0 00000100:00100000:3.0:1789762800.060146:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 71 00000100:00100000:1.0:1789762800.060337:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762800.060385:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014976896 00000100:00100000:0.0:1789762800.060392:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014976896 00000100:00100000:0.0:1789762800.060427:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 21 00000100:00100000:0.0:1789762800.060434:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86f00c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2364:x1876702014976896:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762800.060441:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762800.060451:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03ad2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd567 expref: 23 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.060460:0:5791:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff981e83a46200) 00010000:00010000:0.0:1789762800.060463:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762800.060487:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86f00c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:2364:x1876702014976896:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 54us (150us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762800.060492:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 21 00000100:00100000:1.0:1789762800.061196:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762800.061224:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014977152 00000100:00100000:0.0:1789762800.061229:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014977152 00000100:00100000:0.0:1789762800.061253:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 72 00000100:00100000:0.0:1789762800.061265:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa79180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:13668:x1876702014977152:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:0.0:1789762800.061355:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa79180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:13668:x1876702014977152:12345-192.168.203.48@tcp:35:dd.0 Request processed in 91us (160us total) trans 4294967496 rc 0/0 00000100:00100000:0.0:1789762800.061377:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 72 00000100:00100000:1.0:1789762800.064221:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762800.064255:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014977280 00000100:00100000:2.0:1789762800.064259:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014977280 00002000:00100000:2.0:1789762800.064286:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xf:0x0] (0->1048575) req@ffff981d8aa7b100 x1876702014977280/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:581/0 lens 488/0 e 0 to 0 dl 1789762811 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762800.064294:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d8aa7b100 with x1876702014977280 ext(0->1048575) 00002000:00010000:2.0:1789762800.064298:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d8aa7b100 00000100:00100000:2.0:1789762800.064314:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 303055416562 time 303055526737 state 0x0 ref 1 00000100:00100000:2.0:1789762800.064321:0:6653:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 6: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981d81a45000 gen 0 rate 10 depth 3 token 0 deadline 303055416562 time 303055526737 state 0x0 ref 1 00000100:00100000:2.0:1789762800.064350:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 3 deadline 323368356307 time 323368256307 state 0x1 ref 1 00000100:00100000:2.0:1789762800.064383:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323368356307 time 323368290950 state 0x0 ref 1 00000100:00100000:2.0:1789762800.064392:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2363:x1876702014977280:12345-192.168.203.48@tcp:3:dd.0 00000100:00100000:2.0:1789762800.070000:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2363:x1876702014977280:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5604us (5776us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762800.070009:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 492 00000100:00100000:2.0:1789762800.070017:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323368356307 time 323368290950 state 0x2 ref 0 00002000:00100000:2.0:1789762800.070027:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xf:0x0] (0->1048575) req@ffff981d8aa7b100 x1876702014977280/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:581/0 lens 488/440 e 0 to 0 dl 1789762811 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762800.070034:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d8aa7b100 with x1876702014977280 ext(0->1048575) 00002000:00010000:2.0:1789762800.070039:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d8aa7b100 00000100:00100000:3.0:1789762800.105474:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762800.105536:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014977536 00000100:00100000:2.0:1789762800.105549:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014977536 00000100:00100000:2.0:1789762800.105620:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 73 00000100:00100000:2.0:1789762800.105632:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:13677:x1876702014977536:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:2.0:1789762800.105959:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:13677:x1876702014977536:12345-192.168.203.48@tcp:35:dd.0 Request processed in 330us (497us total) trans 4294967497 rc 0/0 00000100:00100000:2.0:1789762800.105966:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 73 00000100:00100000:1.0:1789762800.106114:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.106194:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014977792 00000100:00100000:3.0:1789762800.106200:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014977792 00000100:00100000:3.0:1789762800.106274:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 242 00000100:00100000:3.0:1789762800.106282:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:13682:x1876702014977792:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762800.106295:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762800.106321:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03af5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd57c expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.106421:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b03afc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.106436:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b03afc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.106448:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b03afc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762800.106458:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x27:0x0]: returning size 16777216 00010000:00010000:3.0:1789762800.106533:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03af5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd57c expref: 27 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.106547:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b03afc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd57c expref: 26 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.106567:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff981e83a46200) 00010000:00010000:3.0:1789762800.106570:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:3.0:1789762800.106573:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa33600, rc 0) 00000100:00100000:3.0:1789762800.106602:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:13682:x1876702014977792:12345-192.168.203.48@tcp:101:dd.0 Request processed in 322us (515us total) trans 4294967498 rc 0/0 00000100:00100000:3.0:1789762800.106607:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 242 00000100:00100000:2.0:1789762800.119394:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762800.119447:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014977920 00000100:00100000:1.0:1789762800.119455:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014977920 00000100:00100000:1.0:1789762800.119496:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 243 00000100:00100000:1.0:1789762800.119504:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e78a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:13683:x1876702014977920:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:1.0:1789762800.119531:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762800.119546:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b03b0a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd583 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.119604:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b03b11 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.119618:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b03b11 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.119631:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b03b11 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762800.119666:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00000004:00200000:1.0:1789762800.119671:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762800.119691:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b03b11 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.119724:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b03b0a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd583 expref: 28 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.119737:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b03b11 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd583 expref: 27 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.119759:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:1.0:1789762800.119766:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762800.119770:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81483600, rc 0) 00000100:00100000:1.0:1789762800.119829:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e78a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:13683:x1876702014977920:12345-192.168.203.48@tcp:101:rm.0 Request processed in 323us (439us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762800.119837:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 243 00000100:00100000:2.0:1789762800.120976:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.121014:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014978176 00000100:00100000:3.0:1789762800.121019:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014978176 00000100:00100000:3.0:1789762800.121044:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 244 00000100:00100000:3.0:1789762800.121051:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:13683:x1876702014978176:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:3.0:1789762800.121059:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762800.121066:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b18 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd58a expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762800.121134:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00010000:00010000:3.0:1789762800.121215:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b18 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd58a expref: 29 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.121225:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:3.0:1789762800.121227:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762800.121230:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b18 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd58a expref: 29 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.121238:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa32000, rc 0) 00000100:00100000:3.0:1789762800.121265:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:13683:x1876702014978176:12345-192.168.203.48@tcp:101:rm.0 Request processed in 215us (291us total) trans 4294967499 rc 301/301 00000100:00100000:3.0:1789762800.121280:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 244 00000100:00100000:0.0:1789762800.122765:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.122804:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014978304 00000100:00100000:3.0:1789762800.122810:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014978304 00000100:00100000:3.0:1789762800.122838:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 74 00000100:00100000:3.0:1789762800.122845:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:13683:x1876702014978304:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:3.0:1789762800.124676:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:13683:x1876702014978304:12345-192.168.203.48@tcp:37:rm.0 Request processed in 1830us (1912us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762800.124681:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 74 00000100:00100000:2.0:1789762800.126007:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.126040:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014978560 00000100:00100000:3.0:1789762800.126044:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014978560 00000100:00100000:3.0:1789762800.126070:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 75 00000100:00100000:3.0:1789762800.126075:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:13683:x1876702014978560:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762800.126217:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:13683:x1876702014978560:12345-192.168.203.48@tcp:35:rm.0 Request processed in 140us (209us total) trans 4294967500 rc 0/0 00000100:00100000:3.0:1789762800.126222:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 75 00000100:00100000:0.0:1789762800.127038:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.127070:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014978816 00000100:00100000:3.0:1789762800.127075:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014978816 00000100:00100000:3.0:1789762800.127126:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 245 00000100:00100000:3.0:1789762800.127137:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8616f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:13683:x1876702014978816:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:3.0:1789762800.127148:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762800.127158:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03b26 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd591 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.127196:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32400/0x416193e012b03b2d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.127215:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32400/0x416193e012b03b2d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.127223:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32400/0x416193e012b03b2d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762800.127231:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00010000:00010000:3.0:1789762800.127277:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33a00/0x416193e012b03b26 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd591 expref: 32 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.127293:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32400/0x416193e012b03b2d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd591 expref: 31 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.127302:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:3.0:1789762800.127305:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1789762800.127308:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d8aa32400, rc 0) 00000100:00100000:3.0:1789762800.127341:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8616f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13683:x1876702014978816:12345-192.168.203.48@tcp:101:rm.0 Request processed in 207us (304us total) trans 4294967501 rc 0/0 00000100:00100000:3.0:1789762800.127347:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 245 00000100:00100000:2.0:1789762800.128148:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.128173:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014978944 00000100:00100000:3.0:1789762800.128178:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014978944 00000100:00100000:3.0:1789762800.128209:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 246 00000100:00100000:3.0:1789762800.128215:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e79180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13683:x1876702014978944:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:3.0:1789762800.128250:0:5806:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7719616512 avail 7715422208 00000100:00100000:3.0:1789762800.128285:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e79180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13683:x1876702014978944:12345-192.168.203.48@tcp:41:rm.0 Request processed in 70us (137us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762800.128294:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 246 00000100:00100000:2.0:1789762800.129108:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.129130:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014979200 00000100:00100000:3.0:1789762800.129136:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014979200 00000100:00100000:3.0:1789762800.129163:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 76 00000100:00100000:3.0:1789762800.129169:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b18450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13683:x1876702014979200:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762800.129250:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b18450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:13683:x1876702014979200:12345-192.168.203.48@tcp:35:rm.0 Request processed in 81us (142us total) trans 4294967502 rc 0/0 00000100:00100000:3.0:1789762800.129256:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 76 00000100:00100000:0.0:1789762800.129923:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.129954:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014979456 00000100:00100000:3.0:1789762800.129958:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014979456 00000100:00100000:3.0:1789762800.129982:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 247 00000100:00100000:3.0:1789762800.129987:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:13683:x1876702014979456:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:3.0:1789762800.130011:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1789762800.130025:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33600/0x416193e012b03afc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd57c expref: 32 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130038:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32200/0x416193e012b03a2a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd505 expref: 31 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130045:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff981e83a46200) 00010000:00010000:3.0:1789762800.130055:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b03b11 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd583 expref: 30 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130062:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:3.0:1789762800.130065:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762800.130075:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03b3b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130119:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03b3b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130129:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03b3b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130148:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03b42 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130155:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03b42 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130160:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03b42 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130223:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03b49 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130229:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03b49 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x27:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130234:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03b49 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x27:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762800.130778:0:5804:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].15, 64 off9152 00000004:00080000:3.0:1789762800.130787:0:5804:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 323434699273ns, next at 324434697363ns, rc = 0 00000040:00080000:3.0:1789762800.130866:0:5804:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].10, 64 off8832 00000004:00080000:3.0:1789762800.130875:0:5804:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 323434787065ns, next at 324434785648ns, rc = 0 00010000:00010000:3.0:1789762800.130930:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03b50 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130947:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03b50 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x27:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130953:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03b50 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x27:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130961:0:5804:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03b50 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x27:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130969:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03b50 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x27:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130976:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03b50 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x27:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.130999:0:5804:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762800.131015:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03b49 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x27:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131021:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03b49 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x27:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131026:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03b49 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x27:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.131027:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762800.131033:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762800.131036:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03b50 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x27:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131039:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03b49 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x27:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.131055:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.131057:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03b49 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x27:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.131061:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03b50 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x27:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.131071:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03b50 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x27:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131079:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff981e9272c500) 00010000:00010000:3.0:1789762800.131083:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789762800.131108:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03b49 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x27:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.131113:0:8466:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff981e9272c500) 00010000:00010000:1.0:1789762800.131118:0:8466:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789762800.131119:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32c00/0x416193e012b03b49 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x27:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.131121:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03b50 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x27:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.131137:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32600/0x416193e012b03b50 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x27:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131149:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03b3b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.131151:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762800.131155:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762800.131159:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762800.131160:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03b3b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131166:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03b3b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131170:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.131172:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03b3b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131176:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03b3b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131182:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:3.0:1789762800.131184:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762800.131186:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03b3b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131189:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33000/0x416193e012b03b3b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762800.131206:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b353d637 reply state 00000000d78ff01b transno 4294967503 ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03b42 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.131213:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762800.131247:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13683:x1876702014979456:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1259us (1326us total) trans 4294967503 rc 0/0 00000100:00100000:3.0:1789762800.131253:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 247 00010000:00010000:3.0:1789762800.131913:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03b42 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131920:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03b42 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131924:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03b42 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131928:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.131930:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03b42 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131934:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03b42 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131940:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x77289514 (ffff981e9272ce00) 00010000:00010000:3.0:1789762800.131942:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762800.131944:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03b42 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.131948:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33200/0x416193e012b03b42 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762800.133453:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.133478:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014979712 00000100:00100000:3.0:1789762800.133483:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014979712 00000100:00100000:3.0:1789762800.133545:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 248 00000100:00100000:3.0:1789762800.133552:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81396a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:13683:x1876702014979712:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:3.0:1789762800.133573:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762800.133585:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33800/0x416193e012b03a62 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd51a expref: 30 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.133601:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff981e83a46000) 00010000:00010000:3.0:1789762800.133607:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762800.133615:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b57 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.133621:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b57 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.133627:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b57 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.133635:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b03b5e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.133640:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b03b5e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.133645:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b03b5e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.133686:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03b65 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.133698:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03b65 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x28:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.133703:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03b65 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x28:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762800.133967:0:8165:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].11, 64 off8896 00000040:00080000:3.0:1789762800.133980:0:8165:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].16, 64 off9216 00010000:00010000:3.0:1789762800.134009:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03b6c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134016:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03b6c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134022:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03b6c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134027:0:8165:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03b6c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134032:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03b6c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134035:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03b6c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134048:0:8165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762800.134056:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03b65 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x28:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134060:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03b65 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x28:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134073:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03b65 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x28:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134077:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.134079:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03b65 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x28:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134083:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03b65 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x28:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134108:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762800.134116:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03b6c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134117:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff981e9272c200) 00010000:00010000:3.0:1789762800.134126:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762800.134130:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.134132:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03b65 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x28:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134135:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03b6c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134153:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03b6c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134153:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32800/0x416193e012b03b65 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x28:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134176:0:6663:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff981e9272c200) 00010000:00010000:3.0:1789762800.134178:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b57 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134179:0:6663:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762800.134184:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03b6c lrc: 1/0,0 mode: --/PW res: [0x200000401:0x28:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134189:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b57 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134192:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32e00/0x416193e012b03b6c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x28:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134197:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b57 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134204:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.134206:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b57 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134209:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762800.134212:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762800.134213:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762800.134221:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b57 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134232:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:3.0:1789762800.134234:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762800.134236:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b57 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.134240:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32000/0x416193e012b03b57 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762800.134247:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e0652e97 reply state 00000000614bd8cc transno 4294967504 ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b03b5e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.134252:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762800.134286:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81396a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13683:x1876702014979712:12345-192.168.203.48@tcp:36:rm.0 Request processed in 735us (835us total) trans 4294967504 rc 0/0 00000100:00100000:3.0:1789762800.134292:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 248 00010000:00010000:0.0:1789762800.134794:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b03b5e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134801:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b03b5e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134806:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b03b5e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134810:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa33e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762800.134812:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b03b5e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134817:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b03b5e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134823:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x992ad74f (ffff981e9272c100) 00010000:00010000:0.0:1789762800.134825:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762800.134827:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b03b5e lrc: 2/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.134831:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33e00/0x416193e012b03b5e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x25:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762800.135289:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.135311:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014979968 00000100:00100000:3.0:1789762800.135315:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014979968 00000100:00100000:3.0:1789762800.135377:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 77 00000100:00100000:3.0:1789762800.135384:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:13683:x1876702014979968:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762800.135475:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13683:x1876702014979968:12345-192.168.203.48@tcp:35:rm.0 Request processed in 91us (186us total) trans 4294967505 rc 0/0 00000100:00100000:3.0:1789762800.135483:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 77 00000100:00100000:2.0:1789762800.136219:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.136249:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014980224 00000100:00100000:3.0:1789762800.136253:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014980224 00000100:00100000:3.0:1789762800.136276:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 249 00000100:00100000:3.0:1789762800.136288:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13683:x1876702014980224:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:3.0:1789762800.136306:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1789762800.136315:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32400/0x416193e012b03b2d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd591 expref: 31 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136328:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa32a00/0x416193e012b03a0e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4fe expref: 30 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136335:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e83a46d00) 00010000:00010000:3.0:1789762800.136344:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d8aa33400/0x416193e012b039ac lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd4e9 expref: 29 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136351:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e83a46e00) 00010000:00010000:3.0:1789762800.136355:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762800.136382:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b73 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136389:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b73 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136394:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b73 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136401:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b7a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136406:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b7a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136411:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b7a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136457:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03b81 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136464:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03b81 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x25:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136469:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03b81 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x25:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762800.136712:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00010000:00010000:3.0:1789762800.136720:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03b81 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x25:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136725:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03b81 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x25:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136729:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03b81 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x25:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136733:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.136735:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03b81 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x25:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136739:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03b81 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x25:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136745:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff981e9272c000) 00010000:00010000:3.0:1789762800.136746:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789762800.136749:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03b81 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x25:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136768:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03b81 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x25:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136786:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b73 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136794:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b73 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136801:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b73 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136807:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.136810:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b73 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136814:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b73 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136821:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:3.0:1789762800.136823:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762800.136825:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b73 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.136829:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b73 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762800.136837:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009706cfbc reply state 00000000844e6303 transno 4294967506 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b7a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.136841:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762800.136878:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:13683:x1876702014980224:12345-192.168.203.48@tcp:36:rm.0 Request processed in 590us (661us total) trans 4294967506 rc 0/0 00000100:00100000:3.0:1789762800.136883:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 249 00010000:00010000:3.0:1789762800.137336:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b7a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.137342:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b7a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.137347:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b7a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.137350:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.137352:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b7a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.137356:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b7a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.137378:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272cc00) 00010000:00010000:3.0:1789762800.137379:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762800.137381:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b7a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.137385:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b7a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762800.420829:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.420883:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014980352 00000100:00100000:3.0:1789762800.420898:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014980352 00000100:00100000:3.0:1789762800.420964:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 250 00000100:00100000:3.0:1789762800.420978:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:13694:x1876702014980352:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:3.0:1789762800.421014:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762800.421038:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b88 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd598 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.421130:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b8f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.421144:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b8f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.421155:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b8f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762800.421194:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762800.421197:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762800.421216:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b8f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.421239:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03b88 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd598 expref: 31 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.421252:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b8f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd598 expref: 30 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.421260:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:3.0:1789762800.421263:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762800.421266:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb4000, rc 0) 00000100:00100000:3.0:1789762800.421330:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:13694:x1876702014980352:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 356us (509us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762800.421344:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 250 00000100:00100000:3.0:1789762800.422289:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762800.422331:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014980608 00000100:00100000:0.0:1789762800.422337:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014980608 00000100:00100000:0.0:1789762800.422391:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 251 00000100:00100000:0.0:1789762800.422403:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:13694:x1876702014980608:12345-192.168.203.48@tcp:36:mkdir.0 00010000:00010000:0.0:1789762800.422437:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762800.422455:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b03b8f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd598 expref: 30 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.422474:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:0.0:1789762800.422483:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762800.422553:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b03b96 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.422561:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b03b96 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.422569:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b03b96 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.422580:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03b9d lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.422586:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03b9d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.422592:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03b9d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762800.423352:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x29:0x0]: returning size 16384 00010000:00010000:0.0:1789762800.423376:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b03b96 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.423382:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b03b96 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.423388:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b03b96 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.423393:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aae00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762800.423403:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b03b96 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.423410:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b03b96 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.423418:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762800.423420:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762800.423423:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b03b96 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.423428:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b03b96 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762800.423439:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f230a8e3 reply state 000000002f8dc24c transno 4294967507 ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03b9d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762800.423446:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762800.423493:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13694:x1876702014980608:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 1094us (1208us total) trans 4294967507 rc 0/0 00000100:00100000:0.0:1789762800.423498:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 251 00010000:00010000:3.0:1789762800.423967:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03b9d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.423976:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03b9d lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.423981:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03b9d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.423985:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aac00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.423988:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03b9d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.423992:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03b9d lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.424000:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3a00) 00010000:00010000:3.0:1789762800.424002:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762800.424005:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03b9d lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.424009:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b03b9d lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762800.429674:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.429759:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014980736 00000100:00100000:3.0:1789762800.429767:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014980736 00000100:00100000:3.0:1789762800.429815:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 252 00000100:00100000:3.0:1789762800.429828:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:13695:x1876702014980736:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762800.429851:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762800.429865:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03ba4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd59f expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.429913:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03bab lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.429922:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03bab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.429930:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03bab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762800.429949:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762800.429951:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762800.429962:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03bab lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.429977:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03ba4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd59f expref: 32 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.429988:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03bab lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd59f expref: 31 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.429994:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:3.0:1789762800.429997:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762800.429999:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb5600, rc 0) 00000100:00100000:3.0:1789762800.430034:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13695:x1876702014980736:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 211us (363us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762800.430039:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 252 00000100:00100000:0.0:1789762800.431158:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762800.431188:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014980992 00000100:00100000:1.0:1789762800.431194:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014980992 00000100:00100000:1.0:1789762800.431226:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 253 00000100:00100000:1.0:1789762800.431234:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13695:x1876702014980992:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762800.431242:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762800.431244:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1789762800.431258:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b03bab lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd59f expref: 31 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431273:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:1.0:1789762800.431282:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762800.431293:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b03bb2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5a6 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431398:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b03bb9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431406:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b03bb9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431412:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b03bb9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431424:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b03bc0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431429:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b03bc0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431434:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b03bc0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431497:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03bc7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431503:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03bc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431508:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03bc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762800.431514:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x29:0x0]: returning size 16384 00010000:00010000:1.0:1789762800.431522:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03bc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431527:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03bc7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431532:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03bc7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431535:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762800.431538:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03bc7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431542:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03bc7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431548:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981ec2179400) 00010000:00010000:1.0:1789762800.431550:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762800.431552:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03bc7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431565:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03bc7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431581:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b03bb9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431585:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b03bb9 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431590:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b03bb9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431594:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762800.431596:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b03bb9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431600:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b03bb9 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431605:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179600) 00010000:00010000:1.0:1789762800.431606:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762800.431608:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b03bb9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431612:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b03bb9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431618:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b03bc0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431623:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b03bc0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431631:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b03bc0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431635:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762800.431636:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b03bc0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431640:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b03bc0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431645:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179700) 00010000:00010000:1.0:1789762800.431647:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762800.431649:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b03bc0 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431652:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b03bc0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431670:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b03bb2 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5a6 expref: 31 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431677:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179600) 00010000:00010000:1.0:1789762800.431679:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762800.431681:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b03bb2 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5a6 expref: 31 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.431692:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81483600, rc 0) 00000100:00100000:1.0:1789762800.431722:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:13695:x1876702014980992:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 490us (567us total) trans 0 rc 301/301 00000100:00100000:1.0:1789762800.431736:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 253 00000100:00100000:2.0:1789762800.432505:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762800.432545:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014981120 00000100:00100000:0.0:1789762800.432551:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014981120 00000100:00100000:0.0:1789762800.432580:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 254 00000100:00100000:0.0:1789762800.432589:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:13695:x1876702014981120:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762800.432597:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762800.432606:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b03bce lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5ad expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.432656:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b03bd5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.432664:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b03bd5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.432671:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b03bd5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762800.432686:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762800.432688:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762800.432698:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b03bd5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.432712:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b03bce lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5ad expref: 32 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.432722:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b03bd5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5ad expref: 31 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.432728:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762800.432730:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762800.432732:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aa400, rc 0) 00000100:00100000:0.0:1789762800.432762:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13695:x1876702014981120:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 176us (259us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762800.432767:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 254 00000100:00100000:0.0:1789762800.433602:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762800.433639:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014981376 00000100:00100000:1.0:1789762800.433644:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014981376 00000100:00100000:1.0:1789762800.433673:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 255 00000100:00100000:1.0:1789762800.433681:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:13695:x1876702014981376:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762800.433689:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762800.433698:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b03bdc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5b4 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433726:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b03be3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433733:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b03be3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433739:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b03be3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433749:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b03bea lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433754:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b03bea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433759:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b03bea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762800.433803:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x29:0x0]: returning size 16384 00010000:00010000:1.0:1789762800.433879:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b03be3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433894:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b03be3 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433899:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b03be3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433902:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762800.433905:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b03be3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433911:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b03be3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433917:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:1.0:1789762800.433919:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762800.433921:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b03be3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433925:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b03be3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433933:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b03bea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433938:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b03bea lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433943:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b03bea lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433946:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762800.433952:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b03bea lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433957:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b03bea lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433962:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179500) 00010000:00010000:1.0:1789762800.433964:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762800.433966:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b03bea lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433969:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b03bea lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433991:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b03bdc lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5b4 expref: 33 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.433997:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:1.0:1789762800.433998:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789762800.434001:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b03bdc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5b4 expref: 33 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762800.434010:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81482800, rc 0) 00000100:00100000:1.0:1789762800.434039:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:13695:x1876702014981376:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 359us (438us total) trans 4294967508 rc 301/301 00000100:00100000:1.0:1789762800.434045:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 255 00000100:00100000:0.0:1789762800.434901:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.434932:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014981632 00000100:00100000:3.0:1789762800.434938:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014981632 00000100:00100000:3.0:1789762800.434966:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 256 00000100:00100000:3.0:1789762800.434974:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:13695:x1876702014981632:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:3.0:1789762800.435011:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03bf8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.435020:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03bf8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.435027:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03bf8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762800.435182:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003f45b0b8 reply state 00000000e3a28298 transno 4294967509 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03bf8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.435192:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762800.435199:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x29:0x0]: returning size 16384 00000100:00100000:3.0:1789762800.435254:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:13695:x1876702014981632:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 278us (351us total) trans 4294967509 rc 0/0 00000100:00100000:3.0:1789762800.435263:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 256 00010000:00010000:3.0:1789762800.435760:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03bf8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.435767:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03bf8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.435772:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03bf8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.435777:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.435785:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03bf8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.435789:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03bf8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.435796:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981e9272c000) 00010000:00010000:3.0:1789762800.435798:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789762800.435800:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03bf8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.435804:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03bf8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762800.435927:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.436001:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014981888 00000100:00100000:3.0:1789762800.436006:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014981888 00000100:00100000:3.0:1789762800.436034:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 78 00000100:00100000:3.0:1789762800.436041:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:13695:x1876702014981888:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:3.0:1789762800.436175:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:13695:x1876702014981888:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 134us (247us total) trans 4294967510 rc 0/0 00000100:00100000:3.0:1789762800.436182:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 78 00000100:00100000:1.0:1789762800.439559:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762800.439600:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014982016 00000100:00100000:0.0:1789762800.439606:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014982016 00000100:00100000:0.0:1789762800.439640:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 257 00000100:00100000:0.0:1789762800.439648:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:13696:x1876702014982016:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:0.0:1789762800.439659:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762800.439671:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03bff lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5bb expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439725:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c06 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439735:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c06 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439742:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c06 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439752:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c0d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439757:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c0d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439762:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c0d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439808:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b03c14 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439813:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b03c14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439826:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b03c14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762800.439844:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x29:0x0]: returning size 16384 00000004:00200000:0.0:1789762800.439846:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762800.439855:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b03c14 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439862:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c06 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439867:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c06 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439872:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c06 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439875:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aa600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762800.439877:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c06 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439881:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c06 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439888:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762800.439890:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762800.439892:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c06 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439896:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c06 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439914:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c0d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439919:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c0d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439923:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c0d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439926:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aaa00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762800.439929:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c0d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439932:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c0d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439938:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3c00) 00010000:00010000:0.0:1789762800.439939:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762800.439941:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c0d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439945:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c0d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439964:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03bff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5bb expref: 36 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439972:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b03c14 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5bb expref: 35 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.439987:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981ea0aa3700) 00010000:00010000:0.0:1789762800.439989:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762800.439991:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880ab000, rc 0) 00000100:00100000:0.0:1789762800.440024:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:13696:x1876702014982016:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 377us (468us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762800.440029:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 257 00000100:00100000:0.0:1789762800.441005:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762800.441052:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014982272 00000100:00100000:1.0:1789762800.441058:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014982272 00000100:00100000:0.0:1789762800.441140:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 258 00000100:00100000:0.0:1789762800.441153:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e849e3050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:13696:x1876702014982272:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:0.0:1789762800.441175:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762800.441188:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b03c14 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x29:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5bb expref: 35 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.441204:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981ea0aa3700) 00010000:00010000:0.0:1789762800.441217:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762800.441234:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b03c1b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.441245:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b03c1b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.441252:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b03c1b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762800.441407:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000bcc4374e reply state 00000000a70218d9 transno 4294967511 ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b03c1b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762800.441427:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762800.441432:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x29:0x0]: returning size 16384 00000100:00100000:0.0:1789762800.441473:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e849e3050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:13696:x1876702014982272:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 324us (467us total) trans 4294967511 rc 0/0 00000100:00100000:0.0:1789762800.441479:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 258 00010000:00010000:3.0:1789762800.442012:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b03c1b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.442024:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b03c1b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.442032:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b03c1b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.442036:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.442046:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b03c1b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.442052:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b03c1b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.442062:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981ea0aa3800) 00010000:00010000:3.0:1789762800.442064:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762800.442067:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b03c1b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.442073:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b03c1b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x29:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762800.462729:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.462862:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014982400 00000100:00100000:3.0:1789762800.462876:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014982400 00000100:00100000:3.0:1789762800.462940:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 259 00000100:00100000:3.0:1789762800.462950:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:13702:x1876702014982400:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762800.462977:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762800.463006:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03c22 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5c2 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463072:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03c29 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463111:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03c29 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463123:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03c29 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463137:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03c30 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463151:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03c30 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463159:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03c30 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463283:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b03c37 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463289:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b03c37 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463294:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b03c37 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762800.463312:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x29:0x0]: returning size 16384 00000004:00200000:3.0:1789762800.463315:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762800.463332:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b03c37 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463342:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03c29 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463347:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03c29 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463352:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03c29 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463370:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.463374:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03c29 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463386:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03c29 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463397:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:3.0:1789762800.463399:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762800.463403:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03c29 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463407:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b03c29 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463416:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03c30 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463419:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03c30 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463423:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03c30 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463426:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.463428:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03c30 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463431:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03c30 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463435:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272c000) 00010000:00010000:3.0:1789762800.463437:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762800.463438:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03c30 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463449:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b03c30 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463478:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b03c22 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5c2 expref: 37 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463487:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b03c37 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5c2 expref: 36 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.463493:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981e9272ce00) 00010000:00010000:3.0:1789762800.463495:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789762800.463497:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb4e00, rc 0) 00000100:00100000:3.0:1789762800.463557:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:13702:x1876702014982400:12345-192.168.203.48@tcp:101:dd.500 Request processed in 608us (835us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762800.463562:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 259 00000100:00100000:1.0:1789762800.464561:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762800.464609:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014982656 00000100:00100000:0.0:1789762800.464618:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014982656 00000100:00100000:0.0:1789762800.464663:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 260 00000100:00100000:0.0:1789762800.464677:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa79500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:13702:x1876702014982656:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762800.464694:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762800.464710:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b03c3e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5c9 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.464829:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03c45 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.464841:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03c45 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.464849:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03c45 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.464858:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c4c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.464864:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c4c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.464880:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c4c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762800.465283:0:8165:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x11:0x0], index 0: 0 00010000:00010000:0.0:1789762800.465423:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c53 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.465434:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.465443:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c53 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.465475:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03c45 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.465480:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03c45 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.465485:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03c45 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.465489:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762800.465493:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03c45 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.465498:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03c45 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.465507:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981e9272ce00) 00010000:00010000:0.0:1789762800.465509:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762800.465518:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03c45 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.465522:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b03c45 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762800.465532:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b15459c9 reply state 00000000f9675317 transno 4294967512 ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c4c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762800.465538:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762800.465559:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b03c3e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5c9 expref: 39 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.465568:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c53 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5c9 expref: 38 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762800.465573:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff981ea0aa3c00) 00010000:00010000:0.0:1789762800.465575:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762800.465577:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aa600, rc 0) 00000100:00100000:0.0:1789762800.465632:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa79500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:13702:x1876702014982656:12345-192.168.203.48@tcp:101:dd.500 Request processed in 961us (1073us total) trans 4294967512 rc 0/0 00000100:00100000:0.0:1789762800.465637:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 260 00010000:00010000:3.0:1789762800.466281:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c4c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.466291:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c4c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.466297:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c4c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.466304:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aaa00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762800.466309:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c4c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.466315:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c4c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.466338:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x4009f790 (ffff981ea0aa3700) 00010000:00010000:3.0:1789762800.466341:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762800.466345:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c4c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.466351:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b03c4c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762800.468824:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762800.468872:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014982784 00000100:00100000:2.0:1789762800.468878:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014982784 00000100:00100000:2.0:1789762800.468920:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323270343035 time 323270400528 state 0x0 ref 1 00000100:00100000:2.0:1789762800.468934:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323320400528 time 323270400528 state 0x1 ref 1 00000100:00100000:2.0:1789762800.468957:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 323320400528 time 323772864512 state 0x0 ref 1 00000100:00100000:2.0:1789762800.468971:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2364:x1876702014982784:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762800.469014:0:14913:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789762800.469021:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.469041:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=323s v=(0 0) 00010000:00010000:2.0:1789762800.469045:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762800.469050:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.469060:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762800.496582:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967805 is committed 00000001:00080000:0.0:1789762800.496613:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967807 is committed 00002000:00010000:3.0:1789762800.496628:0:14913:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:3.0:1789762800.496635:0:14913:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb mtime from disk: 0 -> 1789762800 00002000:00010000:3.0:1789762800.496638:0:14913:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb ctime from disk: 0 -> 1789762800 00002000:00010000:3.0:1789762800.496640:0:14913:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:3.0:1789762800.496657:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.496670:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762800.496672:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967809 is committed 00010000:00010000:3.0:1789762800.496684:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.496690:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.496696:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.496704:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.496738:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.496744:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.496858:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014982784:12345-192.168.203.48@tcp:4:dd.500 Request processed in 27883us (28029us total) trans 4294967809 rc 0/0 00000100:00100000:3.0:1789762800.496869:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 493 00000100:00100000:3.0:1789762800.496882:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 323320400528 time 323772864512 state 0x2 ref 0 00000100:00100000:2.0:1789762800.511482:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.511537:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014982912 00000100:00100000:3.0:1789762800.511549:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014982912 00000100:00100000:3.0:1789762800.511614:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 323320400528 time 323772864512 state 0x0 ref 1 00000100:00100000:3.0:1789762800.511629:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 323822864512 time 323772864512 state 0x1 ref 1 00000100:00100000:3.0:1789762800.511654:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 323822864512 time 323815562483 state 0x0 ref 1 00000100:00100000:3.0:1789762800.511665:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014982912:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762800.511716:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 1048576, mtime: 0x6aad9cf0, atime: 0x0, ctime: 0x6aad9cf0, blocks: 0x803 00010000:00010000:3.0:1789762800.511724:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03c68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.511757:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=323s v=(0 0) 00010000:00010000:3.0:1789762800.511762:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.511767:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03c68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.511782:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03c68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762800.528657:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967810 is committed 00002000:00010000:3.0:1789762800.528674:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:3.0:1789762800.528680:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:3.0:1789762800.528697:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03c68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.528713:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03c68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.528723:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03c68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.528728:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.528733:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03c68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.528741:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03c68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.528774:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03c68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.528781:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03c68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.528872:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014982912:12345-192.168.203.48@tcp:4:dd.500 Request processed in 17203us (17396us total) trans 4294967810 rc 0/0 00000100:00100000:3.0:1789762800.528882:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 494 00000100:00100000:3.0:1789762800.528894:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 323822864512 time 323815562483 state 0x2 ref 0 00000100:00100000:1.0:1789762800.531646:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.531682:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014983040 00000100:00100000:3.0:1789762800.531692:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014983040 00000100:00100000:3.0:1789762800.531739:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 323822864512 time 323815562483 state 0x0 ref 1 00000100:00100000:3.0:1789762800.531752:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 323865562483 time 323815562483 state 0x1 ref 1 00000100:00100000:3.0:1789762800.531781:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323865562483 time 323835686627 state 0x0 ref 1 00000100:00100000:3.0:1789762800.531795:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014983040:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762800.531839:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 2097152, mtime: 0x6aad9cf0, atime: 0x0, ctime: 0x6aad9cf0, blocks: 0x1007 00010000:00010000:3.0:1789762800.531847:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.531874:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=323s v=(0 0) 00010000:00010000:3.0:1789762800.531881:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.531888:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.531903:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762800.547521:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967811 is committed 00002000:00010000:3.0:1789762800.547532:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:3.0:1789762800.547539:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:3.0:1789762800.547557:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.547569:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.547577:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.547582:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.547587:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.547594:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.547624:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.547630:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.547717:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014983040:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15919us (16066us total) trans 4294967811 rc 0/0 00000100:00100000:3.0:1789762800.547731:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 495 00000100:00100000:3.0:1789762800.547746:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323865562483 time 323835686627 state 0x2 ref 0 00000100:00100000:0.0:1789762800.550582:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.550621:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014983168 00000100:00100000:3.0:1789762800.550630:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014983168 00000100:00100000:3.0:1789762800.550681:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323865562483 time 323835686627 state 0x0 ref 1 00000100:00100000:3.0:1789762800.550699:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323885686627 time 323835686627 state 0x1 ref 1 00010000:00010000:1.0:1789762800.567228:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762800.581907:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323885686627 time 323885797761 state 0x0 ref 1 00000100:00100000:2.0:1789762800.581938:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014983168:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762800.582013:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 3145728, mtime: 0x6aad9cf0, atime: 0x0, ctime: 0x6aad9cf0, blocks: 0x1807 00010000:00010000:2.0:1789762800.582027:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.582048:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=323s v=(0 0) 00010000:00010000:2.0:1789762800.582054:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762800.582063:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.582080:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762800.597561:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967812 is committed 00002000:00010000:2.0:1789762800.597623:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:2.0:1789762800.597627:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:2.0:1789762800.597641:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.597652:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.597660:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.597665:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762800.597670:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.597678:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.597696:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.597702:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762800.597800:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014983168:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15869us (47220us total) trans 4294967812 rc 0/0 00000100:00100000:2.0:1789762800.597818:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 496 00000100:00100000:2.0:1789762800.597829:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323885686627 time 323885797761 state 0x2 ref 0 00000100:00100000:0.0:1789762800.600368:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762800.600445:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014983296 00000100:00100000:1.0:1789762800.600459:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014983296 00000100:00100000:1.0:1789762800.600523:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323885686627 time 323885797761 state 0x0 ref 1 00000100:00100000:1.0:1789762800.600541:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323935797761 time 323885797761 state 0x1 ref 1 00000100:00100000:2.0:1789762800.632037:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323935797761 time 323935926978 state 0x0 ref 1 00000100:00100000:2.0:1789762800.632068:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014983296:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762800.632151:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 4194304, mtime: 0x6aad9cf0, atime: 0x0, ctime: 0x6aad9cf0, blocks: 0x2007 00010000:00010000:2.0:1789762800.632159:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.632176:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=323s v=(0 0) 00010000:00010000:2.0:1789762800.632180:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762800.632187:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.632201:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762800.647348:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967813 is committed 00002000:00010000:3.0:1789762800.647376:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:3.0:1789762800.647385:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:3.0:1789762800.647403:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.647419:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.647442:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.647449:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.647453:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.647461:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.647479:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.647485:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03c7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.647566:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014983296:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15507us (47214us total) trans 4294967813 rc 0/0 00000100:00100000:3.0:1789762800.647576:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 497 00000100:00100000:3.0:1789762800.647587:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323935797761 time 323935926978 state 0x2 ref 0 00000100:00100000:0.0:1789762800.650173:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.650218:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014983424 00000100:00100000:3.0:1789762800.650227:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014983424 00000100:00100000:3.0:1789762800.650276:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323935797761 time 323935926978 state 0x0 ref 1 00000100:00100000:3.0:1789762800.650306:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323985926978 time 323935926978 state 0x1 ref 1 00000100:00100000:3.0:1789762800.682140:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323985926978 time 323986032444 state 0x0 ref 1 00000100:00100000:3.0:1789762800.682173:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014983424:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762800.682244:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 5242880, mtime: 0x6aad9cf0, atime: 0x0, ctime: 0x6aad9cf0, blocks: 0x2807 00010000:00010000:3.0:1789762800.682257:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.682277:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=323s v=(0 0) 00010000:00010000:3.0:1789762800.682283:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.682292:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.682310:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762800.697205:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967814 is committed 00002000:00010000:3.0:1789762800.697225:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:3.0:1789762800.697232:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:3.0:1789762800.697245:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.697256:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.697275:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.697281:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.697285:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.697292:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.697311:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.697317:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.697408:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014983424:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15246us (47238us total) trans 4294967814 rc 0/0 00000100:00100000:3.0:1789762800.697418:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 498 00000100:00100000:3.0:1789762800.697429:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323985926978 time 323986032444 state 0x2 ref 0 00000100:00100000:1.0:1789762800.699983:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.700025:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014983552 00000100:00100000:3.0:1789762800.700036:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014983552 00000100:00100000:3.0:1789762800.700106:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 323985926978 time 323986032444 state 0x0 ref 1 00000100:00100000:3.0:1789762800.700129:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324036032444 time 323986032444 state 0x1 ref 1 00000100:00100000:2.0:1789762800.732223:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324036032444 time 324036118920 state 0x0 ref 1 00000100:00100000:2.0:1789762800.732249:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014983552:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762800.732305:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 6291456, mtime: 0x6aad9cf0, atime: 0x0, ctime: 0x6aad9cf0, blocks: 0x3007 00010000:00010000:2.0:1789762800.732313:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.732351:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:2.0:1789762800.732366:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762800.732372:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762800.732386:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762800.747068:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:3.0:1789762800.747075:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 12295 -> 14343 00000001:00080000:0.0:1789762800.747081:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967815 is committed 00010000:00010000:3.0:1789762800.747117:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.747129:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.747137:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.747144:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.747148:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.747155:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.747171:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.747176:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03c8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.747260:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014983552:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15020us (47279us total) trans 4294967815 rc 0/0 00000100:00100000:3.0:1789762800.747272:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 499 00000100:00100000:3.0:1789762800.747282:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324036032444 time 324036118920 state 0x2 ref 0 00000100:00100000:1.0:1789762800.749592:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762800.749636:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014983680 00000100:00100000:2.0:1789762800.749646:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014983680 00000100:00100000:2.0:1789762800.749693:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324036032444 time 324036118920 state 0x0 ref 1 00000100:00100000:2.0:1789762800.749707:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324086118920 time 324036118920 state 0x1 ref 1 00000100:00100000:3.0:1789762800.782342:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324086118920 time 324086229723 state 0x0 ref 1 00000100:00100000:3.0:1789762800.782387:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014983680:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762800.782465:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 7340032, mtime: 0x6aad9cf0, atime: 0x0, ctime: 0x6aad9cf0, blocks: 0x3807 00010000:00010000:3.0:1789762800.782477:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.782496:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762800.782501:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.782506:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.782521:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762800.797490:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967816 is committed 00002000:00010000:3.0:1789762800.797494:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:3.0:1789762800.797501:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:3.0:1789762800.797515:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.797536:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.797543:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.797549:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.797554:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.797562:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.797580:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.797585:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.797659:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014983680:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15282us (48066us total) trans 4294967816 rc 0/0 00000100:00100000:3.0:1789762800.797667:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 500 00000100:00100000:3.0:1789762800.797679:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324086118920 time 324086229723 state 0x2 ref 0 00000100:00100000:1.0:1789762800.799915:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.799952:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014983808 00000100:00100000:3.0:1789762800.799969:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014983808 00000100:00100000:3.0:1789762800.800023:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324086118920 time 324086229723 state 0x0 ref 1 00000100:00100000:3.0:1789762800.800036:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324136229723 time 324086229723 state 0x1 ref 1 00000100:00100000:3.0:1789762800.832435:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324136229723 time 324136320800 state 0x0 ref 1 00000100:00100000:3.0:1789762800.832473:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014983808:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762800.832552:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 8388608, mtime: 0x6aad9cf0, atime: 0x0, ctime: 0x6aad9cf0, blocks: 0x4007 00010000:00010000:3.0:1789762800.832564:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.832586:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762800.832592:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.832601:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.832619:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762800.847171:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967817 is committed 00002000:00010000:3.0:1789762800.847225:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:3.0:1789762800.847230:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:3.0:1789762800.847241:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.847252:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.847267:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.847273:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.847276:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.847283:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.847297:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.847303:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03c99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.847392:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014983808:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14933us (47475us total) trans 4294967817 rc 0/0 00000100:00100000:3.0:1789762800.847401:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 501 00000100:00100000:3.0:1789762800.847411:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324136229723 time 324136320800 state 0x2 ref 0 00000100:00100000:1.0:1789762800.849835:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.849877:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014983936 00000100:00100000:3.0:1789762800.849885:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014983936 00000100:00100000:3.0:1789762800.849924:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324136229723 time 324136320800 state 0x0 ref 1 00000100:00100000:3.0:1789762800.849943:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324186320800 time 324136320800 state 0x1 ref 1 00000100:00100000:3.0:1789762800.882505:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324186320800 time 324186402324 state 0x0 ref 1 00000100:00100000:3.0:1789762800.882530:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014983936:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762800.882580:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 9437184, mtime: 0x6aad9cf0, atime: 0x0, ctime: 0x6aad9cf0, blocks: 0x4807 00010000:00010000:3.0:1789762800.882587:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.882600:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762800.882603:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.882608:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.882619:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762800.895927:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967818 is committed 00002000:00010000:3.0:1789762800.895981:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:3.0:1789762800.895986:0:6653:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb mtime from disk: 1789762800 -> 1789762801 00002000:00010000:3.0:1789762800.895988:0:6653:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb ctime from disk: 1789762800 -> 1789762801 00002000:00010000:3.0:1789762800.895990:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 18439 -> 20487 00010000:00010000:3.0:1789762800.895999:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.896008:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.896023:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.896028:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.896032:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.896038:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.896051:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.896056:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.896150:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014983936:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13626us (46316us total) trans 4294967818 rc 0/0 00000100:00100000:3.0:1789762800.896158:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 502 00000100:00100000:3.0:1789762800.896168:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324186320800 time 324186402324 state 0x2 ref 0 00000100:00100000:1.0:1789762800.898418:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762800.898478:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014984064 00000100:00100000:2.0:1789762800.898490:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014984064 00000100:00100000:2.0:1789762800.898547:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324186320800 time 324186402324 state 0x0 ref 1 00000100:00100000:2.0:1789762800.898561:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324236402324 time 324186402324 state 0x1 ref 1 00000100:00100000:3.0:1789762800.932612:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324236402324 time 324236509087 state 0x0 ref 1 00000100:00100000:3.0:1789762800.932635:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014984064:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762800.932699:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 10485760, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0x5007 00010000:00010000:3.0:1789762800.932707:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.932720:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762800.932723:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.932728:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.932739:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762800.945916:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967819 is committed 00002000:00010000:3.0:1789762800.945969:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:3.0:1789762800.945975:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:3.0:1789762800.945984:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.945993:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.945999:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.946005:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.946008:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.946027:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.946041:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.946046:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ca7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.946136:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014984064:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13508us (47722us total) trans 4294967819 rc 0/0 00000100:00100000:3.0:1789762800.946145:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 503 00000100:00100000:3.0:1789762800.946154:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324236402324 time 324236509087 state 0x2 ref 0 00000100:00100000:1.0:1789762800.948385:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.948420:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014984192 00000100:00100000:3.0:1789762800.948428:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014984192 00000100:00100000:3.0:1789762800.948471:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324236402324 time 324236509087 state 0x0 ref 1 00000100:00100000:3.0:1789762800.948481:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324286509087 time 324236509087 state 0x1 ref 1 00000100:00100000:3.0:1789762800.982674:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324286509087 time 324286570100 state 0x0 ref 1 00000100:00100000:3.0:1789762800.982698:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014984192:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762800.982750:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 11534336, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0x5807 00010000:00010000:3.0:1789762800.982757:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.982768:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762800.982772:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762800.982776:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.982786:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762800.995477:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967820 is committed 00002000:00010000:3.0:1789762800.995491:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 11534336 -> 12582912 00002000:00010000:3.0:1789762800.995497:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 22535 -> 24583 00010000:00010000:3.0:1789762800.995506:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.995516:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.995522:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.995527:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762800.995530:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.995550:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.995563:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762800.995568:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762800.995628:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014984192:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12937us (47245us total) trans 4294967820 rc 0/0 00000100:00100000:3.0:1789762800.995636:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 504 00000100:00100000:3.0:1789762800.995647:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324286509087 time 324286570100 state 0x2 ref 0 00000100:00100000:2.0:1789762800.997988:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762800.998027:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014984320 00000100:00100000:3.0:1789762800.998035:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014984320 00000100:00100000:3.0:1789762800.998072:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324286509087 time 324286570100 state 0x0 ref 1 00000100:00100000:3.0:1789762800.998082:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324336570100 time 324286570100 state 0x1 ref 1 00000100:00100000:3.0:1789762801.032759:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324336570100 time 324336655867 state 0x0 ref 1 00000100:00100000:3.0:1789762801.032800:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014984320:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:1.0:1789762801.032835:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789762801.032864:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 12582912, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0x6007 00010000:00010000:3.0:1789762801.032872:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.032886:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.032889:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.032894:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.032905:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762801.046462:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967821 is committed 00002000:00010000:3.0:1789762801.046491:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 12582912 -> 13631488 00002000:00010000:3.0:1789762801.046501:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:3.0:1789762801.046514:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.046528:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cb5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.046539:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.046548:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.046554:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.046577:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cb5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.046593:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cb5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.046599:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cb5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.046676:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014984320:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13897us (48688us total) trans 4294967821 rc 0/0 00000100:00100000:3.0:1789762801.046687:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 505 00000100:00100000:3.0:1789762801.046701:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324336570100 time 324336655867 state 0x2 ref 0 00000100:00100000:0.0:1789762801.049309:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762801.049426:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014984448 00000100:00100000:2.0:1789762801.049440:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014984448 00000100:00100000:2.0:1789762801.049510:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324336570100 time 324336655867 state 0x0 ref 1 00000100:00100000:2.0:1789762801.049550:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324386655867 time 324336655867 state 0x1 ref 1 00000100:00100000:3.0:1789762801.082918:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324386655867 time 324386807475 state 0x0 ref 1 00000100:00100000:3.0:1789762801.082960:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014984448:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.083009:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 13631488, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0x6807 00010000:00010000:3.0:1789762801.083016:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cbc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.083030:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.083033:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.083038:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.083061:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762801.098983:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967822 is committed 00002000:00010000:3.0:1789762801.098987:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:3.0:1789762801.098993:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:3.0:1789762801.099003:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.099012:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cbc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.099019:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.099024:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.099028:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.099034:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cbc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.099047:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cbc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.099053:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cbc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.099151:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014984448:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16199us (49842us total) trans 4294967822 rc 0/0 00000100:00100000:3.0:1789762801.099160:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 506 00000100:00100000:3.0:1789762801.099169:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324386655867 time 324386807475 state 0x2 ref 0 00000100:00100000:2.0:1789762801.102285:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.102390:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014984576 00000100:00100000:3.0:1789762801.102403:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014984576 00000100:00100000:3.0:1789762801.102468:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324386655867 time 324386807475 state 0x0 ref 1 00000100:00100000:3.0:1789762801.102479:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324436807475 time 324386807475 state 0x1 ref 1 00000100:00100000:3.0:1789762801.133026:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324436807475 time 324436921079 state 0x0 ref 1 00000100:00100000:3.0:1789762801.133053:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014984576:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.133140:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 14680064, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0x7007 00010000:00010000:3.0:1789762801.133149:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cc3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.133164:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.133168:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.133174:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.133186:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762801.150616:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967823 is committed 00002000:00010000:3.0:1789762801.150640:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:3.0:1789762801.150651:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:3.0:1789762801.150674:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.150700:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cc3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.150733:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.150747:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.150755:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.150772:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cc3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.150813:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cc3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.150820:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cc3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.150899:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014984576:12345-192.168.203.48@tcp:4:dd.500 Request processed in 17854us (48617us total) trans 4294967823 rc 0/0 00000100:00100000:3.0:1789762801.150917:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 507 00000100:00100000:3.0:1789762801.150927:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324436807475 time 324436921079 state 0x2 ref 0 00000100:00100000:2.0:1789762801.154881:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.154934:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014984704 00000100:00100000:3.0:1789762801.154946:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014984704 00000100:00100000:3.0:1789762801.155018:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324436807475 time 324436921079 state 0x0 ref 1 00000100:00100000:3.0:1789762801.155037:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324486921079 time 324436921079 state 0x1 ref 1 00000100:00100000:3.0:1789762801.183207:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324486921079 time 324487098638 state 0x0 ref 1 00000100:00100000:3.0:1789762801.183256:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014984704:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.183328:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 15728640, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0x7807 00010000:00010000:3.0:1789762801.183342:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.183409:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.183415:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.183422:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.183438:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762801.199657:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967824 is committed 00002000:00010000:3.0:1789762801.199683:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:3.0:1789762801.199692:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:3.0:1789762801.199706:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.199722:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.199735:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.199745:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.199751:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.199760:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.199775:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.199781:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.199876:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014984704:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16621us (44994us total) trans 4294967824 rc 0/0 00000100:00100000:3.0:1789762801.199901:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 508 00000100:00100000:3.0:1789762801.199934:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324486921079 time 324487098638 state 0x2 ref 0 00000100:00100000:1.0:1789762801.203325:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762801.203391:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014984832 00000100:00100000:2.0:1789762801.203413:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014984832 00000100:00100000:2.0:1789762801.203472:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324486921079 time 324487098638 state 0x0 ref 1 00000100:00100000:2.0:1789762801.203486:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324537098638 time 324487098638 state 0x1 ref 1 00000100:00100000:3.0:1789762801.233310:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324537098638 time 324537202475 state 0x0 ref 1 00000100:00100000:3.0:1789762801.233337:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014984832:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.233430:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 16777216, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0x8007 00010000:00010000:3.0:1789762801.233439:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cd1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.233453:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.233456:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.233462:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.233474:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762801.250440:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967825 is committed 00002000:00010000:3.0:1789762801.250460:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 16777216 -> 17825792 00002000:00010000:3.0:1789762801.250485:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:3.0:1789762801.250498:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250510:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cd1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250533:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250540:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.250544:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250553:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cd1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250569:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cd1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.250577:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cd1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.250653:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014984832:12345-192.168.203.48@tcp:4:dd.500 Request processed in 17322us (47330us total) trans 4294967825 rc 0/0 00000100:00100000:3.0:1789762801.250669:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 509 00000100:00100000:3.0:1789762801.250684:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324537098638 time 324537202475 state 0x2 ref 0 00000100:00100000:2.0:1789762801.254017:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.254070:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014984960 00000100:00100000:3.0:1789762801.254081:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014984960 00000100:00100000:3.0:1789762801.254201:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324537098638 time 324537202475 state 0x0 ref 1 00000100:00100000:3.0:1789762801.254252:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324587202475 time 324537202475 state 0x1 ref 1 00000100:00100000:1.0:1789762801.283428:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324587202475 time 324587319521 state 0x0 ref 1 00000100:00100000:1.0:1789762801.283462:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014984960:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762801.283537:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 17825792, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0x8807 00010000:00010000:1.0:1789762801.283549:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03cd8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.283569:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:1.0:1789762801.283587:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762801.283595:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03cd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.283618:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03cd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762801.291344:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967512 is committed 00000004:00080000:2.0:1789762801.291493:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 16 committed 00000004:00080000:2.0:1789762801.291515:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 19 committed 00000100:00100000:2.0:1789762801.292012:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789762801.292042:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022165248:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762801.292167:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762801.292308:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762801.292327:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022165248 00000100:00100000:1.0:1789762801.292338:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022165120:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789762801.292338:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022165248 00000100:00080000:0.0:1789762801.292439:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762801.292462:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:2.0:1789762801.292481:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022165248:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789762801.292482:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:2.0:1789762801.292518:0:6647:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:15 count 1 00000100:00100000:0.0:1789762801.292537:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022165120 00000100:00100000:0.0:1789762801.292551:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022165120 00010000:00010000:2.0:1789762801.292559:0:6647:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 3/0,1 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.292585:0:6647:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03aee lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd575 expref: 9 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.292606:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762801.292611:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.292616:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 4/0,1 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789762801.292627:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762801.292655:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00010000:00010000:2.0:1789762801.292660:0:6647:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03aee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd575 expref: 9 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762801.292682:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022165120:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1789762801.292691:0:6647:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03aee lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd575 expref: 9 pid: 6649 timeout: 424 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762801.292716:0:6647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6647:1876702022165376:192.168.203.48@tcp:104: 00002000:00080000:0.0:1789762801.292726:0:10916:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:10 count 1 00002000:00010000:0.0:1789762801.292772:0:10916:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xa:0x0] initial LVB size: 8388608, mtime: 0x6aad9cef, atime: 0x0, ctime: 0x6aad9cef, blocks: 0x1 00010000:00010000:0.0:1789762801.292785:0:10916:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b03ce6 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.292804:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=324s v=(0 0) 00010000:00010000:0.0:1789762801.292810:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xa:0x0].0x0 since=0s v=(0 0) 00000100:00100000:2.0:1789762801.292818:0:6647:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebf4c3e40 going to sleep for 11 seconds 00010000:00010000:0.0:1789762801.292820:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b03ce6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.292860:0:10916:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b03ce6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.292875:0:10916:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b03ce6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.292886:0:10916:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b03ce6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.292930:0:10916:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762801.292963:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762801.292975:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b03ce6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.292990:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762801.292998:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b03ce6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.293010:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b03ce6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.293031:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b03ce6 lrc: 1/0,0 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.293040:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d729e59/0x416193e012b03ce6 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xa:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.293174:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762801.293197:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762801.293207:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762801.293796:0:6647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6647:1876702022165376:192.168.203.48@tcp:104: 00010000:00010000:2.0:1789762801.293841:0:6647:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 3/0,1 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762801.293949:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022165120:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1275us (1473us total) trans 4294967397 rc 0/0 00000100:00100000:0.0:1789762801.293960:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000004:00080000:1.0:1789762801.294012:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981d846f0a80/2, rc 0, transno 4294967397 00000100:00100000:1.0:1789762801.294041:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022165120:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789762801.294168:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762801.294233:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014985088 00000100:00100000:0.0:1789762801.294241:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014985088 00010000:00010000:0.0:1789762801.294281:0:5790:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03aee lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd575 expref: 10 pid: 6649 timeout: 424 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762801.294325:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 60 00000100:00100000:0.0:1789762801.294338:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702014985088:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762801.294376:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762801.294403:0:5790:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03aee lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd575 expref: 10 pid: 6649 timeout: 424 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.294422:0:5790:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03aee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd575 expref: 10 pid: 6649 timeout: 424 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.294444:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000063bcf5bb/0x416193e012b03aee lrc: 0/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd575 expref: 10 pid: 6649 timeout: 424 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.294460:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762801.294465:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762801.294471:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.294496:0:5790:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.294533:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762801.294565:0:6647:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762801.294589:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702014985088:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 251us (418us total) trans 0 rc 0/0 00010000:00010000:2.0:1789762801.294593:0:6647:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.294623:0:6647:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762801.294626:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 60 00010000:00010000:2.0:1789762801.294641:0:6647:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.294676:0:6647:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762801.294845:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762801.294860:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.294880:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762801.294892:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.294912:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.294938:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 1/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.294948:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03cdf lrc: 0/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.294974:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762801.294977:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762801.294979:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762801.295872:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022165248:12345-0@lo:6:osp-syn-0-0.0 Request processed in 3396us (3707us total) trans 4294967827 rc 0/0 00000100:00100000:2.0:1789762801.295885:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000004:00080000:1.0:1789762801.295926:0:3274:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981e98e7b480/2, rc 0, transno 4294967827 00000100:00100000:1.0:1789762801.295946:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022165248:0@lo:6:osp-syn-0-0.0 00002000:00010000:2.0:1789762801.306638:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:2.0:1789762801.306645:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:2.0:1789762801.306658:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03cd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762801.306672:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967826 is committed 00010000:00010000:2.0:1789762801.306674:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03cd8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.306694:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03cd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.306705:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762801.306717:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03cd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.306725:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03cd8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.306754:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03cd8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.306760:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03cd8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762801.306830:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014984960:12345-192.168.203.48@tcp:4:dd.500 Request processed in 23382us (52817us total) trans 4294967826 rc 0/0 00000100:00100000:2.0:1789762801.306839:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 510 00000100:00100000:2.0:1789762801.306851:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324587202475 time 324587319521 state 0x2 ref 0 00000100:00100000:2.0:1789762801.309872:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.309933:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014985216 00000100:00100000:3.0:1789762801.309941:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014985216 00000100:00100000:3.0:1789762801.309983:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324587202475 time 324587319521 state 0x0 ref 1 00000100:00100000:3.0:1789762801.309996:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324637319521 time 324587319521 state 0x1 ref 1 00000100:00100000:3.0:1789762801.333608:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324637319521 time 324637499504 state 0x0 ref 1 00000100:00100000:3.0:1789762801.333639:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014985216:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.333711:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 18874368, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0x9007 00010000:00010000:3.0:1789762801.333721:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ced lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.333756:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.333761:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.333767:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ced lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.333781:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ced lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762801.351607:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967827 is committed 00000001:00080000:2.0:1789762801.351628:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967828 is committed 00002000:00010000:3.0:1789762801.351642:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:3.0:1789762801.351650:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:3.0:1789762801.351664:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ced lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.351678:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ced lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.351686:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ced lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.351693:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.351697:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ced lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.351705:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ced lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.351724:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ced lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.351730:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03ced lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.351820:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014985216:12345-192.168.203.48@tcp:4:dd.500 Request processed in 18190us (41944us total) trans 4294967828 rc 0/0 00000100:00100000:3.0:1789762801.351841:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 511 00000100:00100000:3.0:1789762801.351852:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324637319521 time 324637499504 state 0x2 ref 0 00000100:00100000:2.0:1789762801.354529:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762801.354600:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014985344 00000100:00100000:1.0:1789762801.354612:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014985344 00000100:00100000:1.0:1789762801.354660:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324637319521 time 324637499504 state 0x0 ref 1 00000100:00100000:1.0:1789762801.354674:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324687499504 time 324637499504 state 0x1 ref 1 00000100:00100000:3.0:1789762801.383766:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324687499504 time 324687657288 state 0x0 ref 1 00000100:00100000:3.0:1789762801.383793:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e79500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014985344:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.383850:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 19922944, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0x9807 00010000:00010000:3.0:1789762801.383862:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cf4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.383897:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.383906:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.383911:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.383924:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762801.395899:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967829 is committed 00002000:00010000:3.0:1789762801.395961:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 19922944 -> 20971520 00002000:00010000:3.0:1789762801.395966:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:3.0:1789762801.395982:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cf4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.395993:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cf4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.396014:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.396021:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.396025:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cf4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.396032:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cf4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.396051:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cf4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.396057:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03cf4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.396175:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e79500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014985344:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12386us (41646us total) trans 4294967829 rc 0/0 00000100:00100000:3.0:1789762801.396185:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 512 00000100:00100000:3.0:1789762801.396197:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324687499504 time 324687657288 state 0x2 ref 0 00000100:00100000:2.0:1789762801.398563:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.398601:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014985472 00000100:00100000:3.0:1789762801.398613:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014985472 00000100:00100000:3.0:1789762801.398675:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324687499504 time 324687657288 state 0x0 ref 1 00000100:00100000:3.0:1789762801.398699:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324737657288 time 324687657288 state 0x1 ref 1 00000100:00100000:2.0:1789762801.433873:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324737657288 time 324737760124 state 0x0 ref 1 00000100:00100000:2.0:1789762801.433908:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014985472:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762801.433981:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 20971520, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0xa007 00010000:00010000:2.0:1789762801.434004:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03cfb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.434020:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:2.0:1789762801.434024:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.434030:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03cfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.434043:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03cfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762801.446171:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967830 is committed 00002000:00010000:3.0:1789762801.446188:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:3.0:1789762801.446196:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:3.0:1789762801.446224:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03cfb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.446240:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03cfb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.446251:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03cfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.446258:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.446264:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03cfb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.446275:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03cfb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.446302:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03cfb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.446312:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b03cfb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.446451:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014985472:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12553us (47884us total) trans 4294967830 rc 0/0 00000100:00100000:3.0:1789762801.446463:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 513 00000100:00100000:3.0:1789762801.446474:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324737657288 time 324737760124 state 0x2 ref 0 00000100:00100000:0.0:1789762801.448812:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.448849:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014985600 00000100:00100000:3.0:1789762801.448859:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014985600 00000100:00100000:3.0:1789762801.448915:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324737657288 time 324737760124 state 0x0 ref 1 00000100:00100000:3.0:1789762801.448930:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324787760124 time 324737760124 state 0x1 ref 1 00000100:00100000:3.0:1789762801.483961:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324787760124 time 324787839472 state 0x0 ref 1 00000100:00100000:3.0:1789762801.484000:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014985600:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.484072:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 22020096, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0xa807 00010000:00010000:3.0:1789762801.484081:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d02 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.484136:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.484140:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.484148:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.484162:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762801.496555:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967831 is committed 00002000:00010000:3.0:1789762801.496583:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 22020096 -> 23068672 00002000:00010000:3.0:1789762801.496590:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:3.0:1789762801.496605:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d02 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.496617:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d02 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.496625:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.496631:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.496635:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d02 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.496642:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d02 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.496674:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d02 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.496683:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d02 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.496784:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014985600:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12794us (47969us total) trans 4294967831 rc 0/0 00000100:00100000:3.0:1789762801.496801:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 514 00000100:00100000:3.0:1789762801.496813:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324787760124 time 324787839472 state 0x2 ref 0 00000100:00100000:2.0:1789762801.499196:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762801.499246:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014985728 00000100:00100000:1.0:1789762801.499257:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014985728 00000100:00100000:1.0:1789762801.499305:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324787760124 time 324787839472 state 0x0 ref 1 00000100:00100000:1.0:1789762801.499319:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324837839472 time 324787839472 state 0x1 ref 1 00000100:00100000:3.0:1789762801.534047:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324837839472 time 324837938962 state 0x0 ref 1 00000100:00100000:3.0:1789762801.534074:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014985728:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.534157:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 23068672, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0xb007 00010000:00010000:3.0:1789762801.534167:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d09 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.534185:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.534191:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.534199:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.534218:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762801.546320:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967832 is committed 00002000:00010000:3.0:1789762801.546341:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:3.0:1789762801.546357:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:3.0:1789762801.546373:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d09 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.546384:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d09 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.546409:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.546419:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.546426:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d09 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.546438:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d09 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.546460:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d09 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.546466:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d09 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.546544:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014985728:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12477us (47346us total) trans 4294967832 rc 0/0 00000100:00100000:3.0:1789762801.546554:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 515 00000100:00100000:3.0:1789762801.546570:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324837839472 time 324837938962 state 0x2 ref 0 00000100:00100000:2.0:1789762801.548777:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.548815:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014985856 00000100:00100000:3.0:1789762801.548825:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014985856 00000100:00100000:3.0:1789762801.548875:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324837839472 time 324837938962 state 0x0 ref 1 00000100:00100000:3.0:1789762801.548899:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324887938962 time 324837938962 state 0x1 ref 1 00000100:00100000:3.0:1789762801.584155:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324887938962 time 324888041540 state 0x0 ref 1 00000100:00100000:3.0:1789762801.584191:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014985856:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.584263:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 24117248, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0xb807 00010000:00010000:3.0:1789762801.584274:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d10 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.584296:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.584302:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.584310:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.584326:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762801.591254:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789762801.596678:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 24117248 -> 25165824 00000001:00080000:0.0:1789762801.596679:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967833 is committed 00002000:00010000:3.0:1789762801.596685:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:3.0:1789762801.596699:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d10 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.596710:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d10 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.596729:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.596735:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.596739:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d10 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.596747:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d10 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.596765:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d10 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.596771:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d10 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.596853:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014985856:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12676us (48074us total) trans 4294967833 rc 0/0 00000100:00100000:3.0:1789762801.596862:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 516 00000100:00100000:3.0:1789762801.596873:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324887938962 time 324888041540 state 0x2 ref 0 00000100:00100000:0.0:1789762801.598977:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.599020:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014985984 00000100:00100000:3.0:1789762801.599031:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014985984 00000100:00100000:3.0:1789762801.599076:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324887938962 time 324888041540 state 0x0 ref 1 00000100:00100000:3.0:1789762801.599120:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324938041540 time 324888041540 state 0x1 ref 1 00000100:00100000:3.0:1789762801.634221:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324938041540 time 324938109549 state 0x0 ref 1 00000100:00100000:3.0:1789762801.634256:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014985984:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.634328:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 25165824, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0xc007 00010000:00010000:3.0:1789762801.634337:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d17 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.634370:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.634376:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.634402:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.634423:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762801.646958:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967834 is committed 00002000:00010000:3.0:1789762801.646964:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:3.0:1789762801.646970:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:3.0:1789762801.646982:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d17 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.646992:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d17 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.647012:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.647018:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.647022:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d17 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.647029:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d17 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.647047:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d17 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.647054:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d17 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.647201:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014985984:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12957us (48220us total) trans 4294967834 rc 0/0 00000100:00100000:3.0:1789762801.647215:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 517 00000100:00100000:3.0:1789762801.647227:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324938041540 time 324938109549 state 0x2 ref 0 00000100:00100000:0.0:1789762801.649329:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762801.649383:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014986112 00000100:00100000:1.0:1789762801.649393:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014986112 00000100:00100000:1.0:1789762801.649442:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324938041540 time 324938109549 state 0x0 ref 1 00000100:00100000:1.0:1789762801.649455:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324988109549 time 324938109549 state 0x1 ref 1 00000100:00100000:3.0:1789762801.684335:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324988109549 time 324988227542 state 0x0 ref 1 00000100:00100000:3.0:1789762801.684383:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014986112:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.684468:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 26214400, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0xc807 00010000:00010000:3.0:1789762801.684477:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d1e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.684494:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=324s v=(0 0) 00010000:00010000:3.0:1789762801.684498:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.684503:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.684517:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762801.696160:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967835 is committed 00002000:00010000:3.0:1789762801.696175:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 26214400 -> 27262976 00002000:00010000:3.0:1789762801.696183:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:3.0:1789762801.696201:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d1e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.696213:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d1e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.696221:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.696227:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762801.696231:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d1e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.696251:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d1e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.696271:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d1e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.696277:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d1e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762801.696375:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014986112:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12003us (47043us total) trans 4294967835 rc 0/0 00000100:00100000:3.0:1789762801.696384:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 518 00000100:00100000:3.0:1789762801.696395:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324988109549 time 324988227542 state 0x2 ref 0 00000100:00100000:1.0:1789762801.698494:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.698532:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014986240 00000100:00100000:3.0:1789762801.698543:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014986240 00000100:00100000:3.0:1789762801.698593:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 324988109549 time 324988227542 state 0x0 ref 1 00000100:00100000:3.0:1789762801.698607:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325038227542 time 324988227542 state 0x1 ref 1 00000100:00100000:3.0:1789762801.734425:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325038227542 time 325038315353 state 0x0 ref 1 00000100:00100000:3.0:1789762801.734456:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014986240:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.734545:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 27262976, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0xd007 00010000:00010000:3.0:1789762801.734554:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d25 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.734572:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:3.0:1789762801.734577:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.734586:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.734608:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762801.745895:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967836 is committed 00002000:00010000:2.0:1789762801.745908:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:2.0:1789762801.745914:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:2.0:1789762801.745926:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d25 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.745937:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d25 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.745945:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.745952:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762801.745958:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d25 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.745984:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d25 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.746008:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d25 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.746017:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b03d25 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762801.746114:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014986240:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11642us (47592us total) trans 4294967836 rc 0/0 00000100:00100000:2.0:1789762801.746124:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 519 00000100:00100000:2.0:1789762801.746133:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325038227542 time 325038315353 state 0x2 ref 0 00000100:00100000:0.0:1789762801.748314:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.748371:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014986368 00000100:00100000:3.0:1789762801.748381:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014986368 00000100:00100000:3.0:1789762801.748430:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325038227542 time 325038315353 state 0x0 ref 1 00000100:00100000:3.0:1789762801.748448:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325088315353 time 325038315353 state 0x1 ref 1 00000100:00100000:3.0:1789762801.784488:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325088315353 time 325088377553 state 0x0 ref 1 00000100:00100000:3.0:1789762801.784519:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014986368:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.784588:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 28311552, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0xd807 00010000:00010000:3.0:1789762801.784597:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03d2c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.784634:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:3.0:1789762801.784638:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.784644:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03d2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.784657:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03d2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762801.796881:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967837 is committed 00002000:00010000:1.0:1789762801.796964:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:1.0:1789762801.796969:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 55303 -> 57351 00010000:00010000:1.0:1789762801.796984:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03d2c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.797000:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03d2c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.797008:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03d2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.797013:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762801.797017:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03d2c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.797024:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03d2c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.797043:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03d2c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.797049:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b03d2c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762801.797156:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014986368:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12648us (48841us total) trans 4294967837 rc 0/0 00000100:00100000:1.0:1789762801.797167:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 520 00000100:00100000:1.0:1789762801.797177:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325088315353 time 325088377553 state 0x2 ref 0 00000100:00100000:0.0:1789762801.799469:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762801.799509:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014986496 00000100:00100000:1.0:1789762801.799520:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014986496 00000100:00100000:1.0:1789762801.799570:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325088315353 time 325088377553 state 0x0 ref 1 00000100:00100000:1.0:1789762801.799584:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325138377553 time 325088377553 state 0x1 ref 1 00000100:00100000:3.0:1789762801.834611:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325138377553 time 325138494516 state 0x0 ref 1 00000100:00100000:3.0:1789762801.834651:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014986496:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762801.834731:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 29360128, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0xe007 00010000:00010000:3.0:1789762801.834742:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b03d33 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.834763:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:3.0:1789762801.834768:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762801.834777:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b03d33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762801.834794:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b03d33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762801.847589:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967838 is committed 00002000:00010000:1.0:1789762801.847609:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:1.0:1789762801.847615:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 57351 -> 59399 00010000:00010000:1.0:1789762801.847630:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b03d33 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.847642:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b03d33 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.847649:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b03d33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.847655:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762801.847659:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b03d33 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.847667:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b03d33 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.847686:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b03d33 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.847691:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b03d33 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762801.847779:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014986496:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13142us (48307us total) trans 4294967838 rc 0/0 00000100:00100000:1.0:1789762801.847789:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 521 00000100:00100000:1.0:1789762801.847801:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325138377553 time 325138494516 state 0x2 ref 0 00000100:00100000:0.0:1789762801.850719:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.850772:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014986624 00000100:00100000:3.0:1789762801.850794:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014986624 00000100:00100000:3.0:1789762801.850853:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325138377553 time 325138494516 state 0x0 ref 1 00000100:00100000:3.0:1789762801.850870:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325188494516 time 325138494516 state 0x1 ref 1 00000100:00100000:2.0:1789762801.884708:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325188494516 time 325188597064 state 0x0 ref 1 00000100:00100000:2.0:1789762801.884743:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014986624:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762801.884811:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 30408704, mtime: 0x6aad9cf1, atime: 0x0, ctime: 0x6aad9cf1, blocks: 0xe807 00010000:00010000:2.0:1789762801.884821:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d3a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.884842:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:2.0:1789762801.884867:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762801.884876:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.884893:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762801.896615:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 30408704 -> 31457280 00002000:00010000:2.0:1789762801.896621:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb mtime from disk: 1789762801 -> 1789762802 00002000:00010000:2.0:1789762801.896624:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb ctime from disk: 1789762801 -> 1789762802 00002000:00010000:2.0:1789762801.896626:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 59399 -> 61447 00010000:00010000:2.0:1789762801.896639:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d3a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.896650:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d3a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.896657:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.896663:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762801.896668:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d3a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.896675:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d3a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.896705:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d3a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.896712:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d3a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762801.896788:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2366:x1876702014986624:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12060us (46068us total) trans 4294967839 rc 0/0 00000100:00100000:2.0:1789762801.896797:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 522 00000100:00100000:2.0:1789762801.896808:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325188494516 time 325188597064 state 0x2 ref 0 00000001:00080000:2.0:1789762801.896854:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967839 is committed 00000100:00100000:3.0:1789762801.899119:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762801.899154:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014986752 00000100:00100000:2.0:1789762801.899166:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014986752 00000100:00100000:2.0:1789762801.899215:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325188494516 time 325188597064 state 0x0 ref 1 00000100:00100000:2.0:1789762801.899229:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325238597064 time 325188597064 state 0x1 ref 1 00000100:00100000:1.0:1789762801.934822:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325238597064 time 325238711713 state 0x0 ref 1 00000100:00100000:1.0:1789762801.934856:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014986752:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762801.934937:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 31457280, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0xf007 00010000:00010000:1.0:1789762801.934946:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03d41 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.934964:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:1.0:1789762801.934969:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762801.934976:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03d41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.934990:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03d41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762801.946731:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967840 is committed 00002000:00010000:2.0:1789762801.946757:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:2.0:1789762801.946765:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 61447 -> 63495 00010000:00010000:2.0:1789762801.946781:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03d41 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.946795:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03d41 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.946815:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03d41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.946822:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762801.946826:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03d41 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.946834:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03d41 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.946852:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03d41 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762801.946859:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03d41 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762801.946940:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014986752:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12095us (47819us total) trans 4294967840 rc 0/0 00000100:00100000:2.0:1789762801.946950:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 523 00000100:00100000:2.0:1789762801.946965:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325238597064 time 325238711713 state 0x2 ref 0 00000100:00100000:1.0:1789762801.949052:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762801.949137:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014986880 00000100:00100000:2.0:1789762801.949148:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014986880 00000100:00100000:2.0:1789762801.949195:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325238597064 time 325238711713 state 0x0 ref 1 00000100:00100000:2.0:1789762801.949219:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325288711713 time 325238711713 state 0x1 ref 1 00000100:00100000:1.0:1789762801.984906:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325288711713 time 325288799720 state 0x0 ref 1 00000100:00100000:1.0:1789762801.984942:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014986880:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762801.985000:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 32505856, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0xf807 00010000:00010000:1.0:1789762801.985009:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b03d48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.985025:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:1.0:1789762801.985030:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762801.985035:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b03d48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.985049:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b03d48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762801.996623:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967841 is committed 00002000:00010000:1.0:1789762801.996645:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:1.0:1789762801.996651:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 63495 -> 65543 00010000:00010000:1.0:1789762801.996665:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b03d48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.996676:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b03d48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.996683:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b03d48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.996689:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762801.996694:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b03d48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.996703:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b03d48 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.996720:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b03d48 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762801.996725:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b03d48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762801.996803:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014986880:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11867us (47750us total) trans 4294967841 rc 0/0 00000100:00100000:1.0:1789762801.996813:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 524 00000100:00100000:1.0:1789762801.996824:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325288711713 time 325288799720 state 0x2 ref 0 00000100:00100000:1.0:1789762801.998929:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762801.998974:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014987008 00000100:00100000:3.0:1789762801.998985:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014987008 00000100:00100000:3.0:1789762801.999043:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325288711713 time 325288799720 state 0x0 ref 1 00000100:00100000:3.0:1789762801.999062:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325338799720 time 325288799720 state 0x1 ref 1 00000100:00100000:2.0:1789762802.034994:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325338799720 time 325338887109 state 0x0 ref 1 00000100:00100000:2.0:1789762802.035020:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014987008:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762802.035078:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 33554432, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x10007 00010000:00010000:2.0:1789762802.035110:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d4f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.035125:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:2.0:1789762802.035129:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.035135:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.035149:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762802.048968:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967842 is committed 00002000:00010000:2.0:1789762802.048982:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:2.0:1789762802.048990:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 65543 -> 67591 00010000:00010000:2.0:1789762802.049006:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d4f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.049021:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d4f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.049041:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.049047:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762802.049051:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d4f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.049059:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d4f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.049076:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d4f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.049082:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d4f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762802.049185:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014987008:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14170us (50251us total) trans 4294967842 rc 0/0 00000100:00100000:2.0:1789762802.049194:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 525 00000100:00100000:2.0:1789762802.049204:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325338799720 time 325338887109 state 0x2 ref 0 00000100:00100000:3.0:1789762802.051276:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.051313:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014987136 00000100:00100000:2.0:1789762802.051325:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014987136 00000100:00100000:2.0:1789762802.051390:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325338799720 time 325338887109 state 0x0 ref 1 00000100:00100000:2.0:1789762802.051416:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325388887109 time 325338887109 state 0x1 ref 1 00000100:00100000:2.0:1789762802.085060:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325388887109 time 325388950850 state 0x0 ref 1 00000100:00100000:2.0:1789762802.085108:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b39180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014987136:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762802.085172:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 34603008, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x10807 00010000:00010000:2.0:1789762802.085181:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d56 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.085198:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:2.0:1789762802.085203:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.085208:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.085221:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762802.097309:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967843 is committed 00002000:00010000:2.0:1789762802.097319:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:2.0:1789762802.097325:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 67591 -> 69639 00010000:00010000:2.0:1789762802.097340:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d56 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.097366:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d56 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.097387:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.097393:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762802.097398:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d56 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.097405:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d56 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.097424:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d56 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.097430:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d56 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762802.097513:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b39180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014987136:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12433us (46236us total) trans 4294967843 rc 0/0 00000100:00100000:2.0:1789762802.097522:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 526 00000100:00100000:2.0:1789762802.097534:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325388887109 time 325388950850 state 0x2 ref 0 00000100:00100000:3.0:1789762802.099862:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.099938:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014987264 00000100:00100000:2.0:1789762802.099950:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014987264 00000100:00100000:2.0:1789762802.100000:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325388887109 time 325388950850 state 0x0 ref 1 00000100:00100000:2.0:1789762802.100022:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325438950850 time 325388950850 state 0x1 ref 1 00000100:00100000:1.0:1789762802.135188:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325438950850 time 325439084918 state 0x0 ref 1 00000100:00100000:1.0:1789762802.135211:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014987264:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762802.135264:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 35651584, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x11007 00010000:00010000:1.0:1789762802.135273:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03d5d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.135288:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:1.0:1789762802.135293:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.135299:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03d5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.135312:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03d5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762802.147687:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 35651584 -> 36700160 00000001:00080000:3.0:1789762802.147690:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967844 is committed 00002000:00010000:2.0:1789762802.147693:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 69639 -> 71687 00010000:00010000:2.0:1789762802.147711:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03d5d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.147722:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03d5d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.147730:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03d5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.147736:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762802.147739:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03d5d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.147746:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03d5d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.147762:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03d5d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.147768:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b03d5d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762802.147849:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014987264:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12644us (47991us total) trans 4294967844 rc 0/0 00000100:00100000:2.0:1789762802.147861:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 527 00000100:00100000:2.0:1789762802.147873:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325438950850 time 325439084918 state 0x2 ref 0 00000100:00100000:0.0:1789762802.149884:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762802.149935:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014987392 00000100:00100000:3.0:1789762802.149945:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014987392 00000100:00100000:3.0:1789762802.149995:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325438950850 time 325439084918 state 0x0 ref 1 00000100:00100000:3.0:1789762802.150007:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325489084918 time 325439084918 state 0x1 ref 1 00000100:00100000:2.0:1789762802.185303:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325489084918 time 325489193572 state 0x0 ref 1 00000100:00100000:2.0:1789762802.185336:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014987392:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762802.185429:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 36700160, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x11807 00010000:00010000:2.0:1789762802.185439:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d64 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.185455:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:2.0:1789762802.185460:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.185466:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.185480:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762802.199155:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967845 is committed 00002000:00010000:2.0:1789762802.199163:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 36700160 -> 37748736 00002000:00010000:2.0:1789762802.199170:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 71687 -> 73735 00010000:00010000:2.0:1789762802.199186:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d64 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.199206:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d64 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.199214:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.199219:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762802.199224:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d64 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.199231:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d64 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.199252:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d64 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.199257:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d64 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762802.199365:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014987392:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14026us (49463us total) trans 4294967845 rc 0/0 00000100:00100000:2.0:1789762802.199376:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 528 00000100:00100000:2.0:1789762802.199387:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325489084918 time 325489193572 state 0x2 ref 0 00000100:00100000:3.0:1789762802.201565:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.201607:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014987520 00000100:00100000:2.0:1789762802.201624:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014987520 00000100:00100000:2.0:1789762802.201671:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325489084918 time 325489193572 state 0x0 ref 1 00000100:00100000:2.0:1789762802.201685:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325539193572 time 325489193572 state 0x1 ref 1 00000100:00100000:2.0:1789762802.235369:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325539193572 time 325539246118 state 0x0 ref 1 00000100:00100000:2.0:1789762802.235406:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b38e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014987520:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762802.235482:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 37748736, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x12007 00010000:00010000:2.0:1789762802.235492:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d6b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.235509:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:2.0:1789762802.235513:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.235519:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.235533:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762802.250430:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967846 is committed 00002000:00010000:2.0:1789762802.250445:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:2.0:1789762802.250450:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 73735 -> 75783 00010000:00010000:2.0:1789762802.250463:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d6b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.250475:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d6b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.250490:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.250496:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762802.250500:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d6b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.250508:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d6b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.250526:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d6b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.250533:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d6b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762802.250608:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b38e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014987520:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15214us (49046us total) trans 4294967846 rc 0/0 00000100:00100000:2.0:1789762802.250620:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 529 00000100:00100000:2.0:1789762802.250634:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325539193572 time 325539246118 state 0x2 ref 0 00000100:00100000:3.0:1789762802.252805:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.252852:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014987648 00000100:00100000:2.0:1789762802.252862:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014987648 00000100:00100000:2.0:1789762802.252909:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325539193572 time 325539246118 state 0x0 ref 1 00000100:00100000:2.0:1789762802.252931:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325589246118 time 325539246118 state 0x1 ref 1 00000100:00100000:2.0:1789762802.285487:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325589246118 time 325589367879 state 0x0 ref 1 00000100:00100000:2.0:1789762802.285529:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b38a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014987648:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762802.285614:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 38797312, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x12807 00010000:00010000:2.0:1789762802.285626:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d72 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.285647:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:2.0:1789762802.285652:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.285660:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.285679:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762802.299400:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967847 is committed 00002000:00010000:1.0:1789762802.299415:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:1.0:1789762802.299423:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 75783 -> 77831 00010000:00010000:1.0:1789762802.299438:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d72 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.299450:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d72 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.299458:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.299464:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.299469:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d72 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.299477:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d72 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.299497:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d72 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.299508:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b03d72 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.299588:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b38a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014987648:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14076us (46785us total) trans 4294967847 rc 0/0 00000100:00100000:1.0:1789762802.299599:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 530 00000100:00100000:1.0:1789762802.299610:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325589246118 time 325589367879 state 0x2 ref 0 00000100:00100000:1.0:1789762802.301766:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762802.301814:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014987776 00000100:00100000:3.0:1789762802.301828:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014987776 00000100:00100000:3.0:1789762802.301889:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325589246118 time 325589367879 state 0x0 ref 1 00000100:00100000:3.0:1789762802.301901:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325639367879 time 325589367879 state 0x1 ref 1 00000100:00100000:2.0:1789762802.335604:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325639367879 time 325639494388 state 0x0 ref 1 00000100:00100000:2.0:1789762802.335637:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc556d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014987776:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762802.335705:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 39845888, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x13007 00010000:00010000:2.0:1789762802.335715:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d79 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.335753:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:2.0:1789762802.335758:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.335764:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.335778:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762802.349083:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967848 is committed 00002000:00010000:2.0:1789762802.349116:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 39845888 -> 40894464 00002000:00010000:2.0:1789762802.349123:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 77831 -> 79879 00010000:00010000:2.0:1789762802.349138:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d79 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.349150:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d79 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.349158:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.349164:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762802.349168:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d79 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.349177:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d79 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.349208:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d79 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.349214:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d79 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762802.349302:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc556d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014987776:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13677us (47533us total) trans 4294967848 rc 0/0 00000100:00100000:2.0:1789762802.349313:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 531 00000100:00100000:2.0:1789762802.349325:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325639367879 time 325639494388 state 0x2 ref 0 00000100:00100000:3.0:1789762802.351644:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.351724:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014987904 00000100:00100000:2.0:1789762802.351735:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014987904 00000100:00100000:2.0:1789762802.351788:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325639367879 time 325639494388 state 0x0 ref 1 00000100:00100000:2.0:1789762802.351802:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325689494388 time 325639494388 state 0x1 ref 1 00000100:00100000:2.0:1789762802.385654:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325689494388 time 325689548850 state 0x0 ref 1 00000100:00100000:2.0:1789762802.385674:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b38000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014987904:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762802.385730:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 40894464, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x13807 00010000:00010000:2.0:1789762802.385738:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d80 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.385769:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:2.0:1789762802.385774:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.385780:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.385799:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762802.398507:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967849 is committed 00002000:00010000:1.0:1789762802.398519:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:1.0:1789762802.398526:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 79879 -> 81927 00010000:00010000:1.0:1789762802.398539:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d80 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.398550:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d80 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.398557:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.398562:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.398566:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d80 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.398574:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d80 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.398594:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d80 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.398600:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03d80 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.398668:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b38000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014987904:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12997us (47029us total) trans 4294967849 rc 0/0 00000100:00100000:1.0:1789762802.398677:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 532 00000100:00100000:1.0:1789762802.398687:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325689494388 time 325689548850 state 0x2 ref 0 00000100:00100000:3.0:1789762802.400744:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.400794:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014988032 00000100:00100000:2.0:1789762802.400803:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014988032 00000100:00100000:2.0:1789762802.400848:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325689494388 time 325689548850 state 0x0 ref 1 00000100:00100000:2.0:1789762802.400862:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325739548850 time 325689548850 state 0x1 ref 1 00000100:00100000:1.0:1789762802.435759:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325739548850 time 325739647438 state 0x0 ref 1 00000100:00100000:1.0:1789762802.435801:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b39500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014988032:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762802.435889:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 41943040, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x14007 00010000:00010000:1.0:1789762802.435898:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d87 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.435917:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:1.0:1789762802.435921:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.435927:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.435940:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762802.449328:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967850 is committed 00002000:00010000:1.0:1789762802.449374:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 41943040 -> 42991616 00002000:00010000:1.0:1789762802.449380:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 81927 -> 83975 00010000:00010000:1.0:1789762802.449397:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d87 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.449409:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d87 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.449416:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.449422:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.449426:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d87 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.449435:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d87 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.449454:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d87 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.449460:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d87 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.449560:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b39500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014988032:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13774us (48816us total) trans 4294967850 rc 0/0 00000100:00100000:1.0:1789762802.449573:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 533 00000100:00100000:1.0:1789762802.449586:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325739548850 time 325739647438 state 0x2 ref 0 00000100:00100000:3.0:1789762802.451922:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762802.451969:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014988160 00000100:00100000:1.0:1789762802.451980:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014988160 00000100:00100000:1.0:1789762802.452048:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325739548850 time 325739647438 state 0x0 ref 1 00000100:00100000:1.0:1789762802.452063:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325789647438 time 325739647438 state 0x1 ref 1 00000100:00100000:2.0:1789762802.485879:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325789647438 time 325789765561 state 0x0 ref 1 00000100:00100000:2.0:1789762802.485913:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014988160:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762802.485993:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 42991616, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x14807 00010000:00010000:2.0:1789762802.486002:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03d8e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.486020:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:2.0:1789762802.486024:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762802.486030:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03d8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.486060:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03d8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762802.498450:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967851 is committed 00002000:00010000:1.0:1789762802.498521:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 42991616 -> 44040192 00002000:00010000:1.0:1789762802.498527:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 83975 -> 86023 00010000:00010000:1.0:1789762802.498539:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03d8e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.498548:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03d8e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.498556:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03d8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.498561:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.498565:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03d8e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.498572:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03d8e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.498592:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03d8e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.498598:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03d8e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.498668:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014988160:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12767us (46753us total) trans 4294967851 rc 0/0 00000100:00100000:1.0:1789762802.498676:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 534 00000100:00100000:1.0:1789762802.498685:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325789647438 time 325789765561 state 0x2 ref 0 00000100:00100000:0.0:1789762802.501149:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.501206:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014988288 00000100:00100000:2.0:1789762802.501217:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014988288 00000100:00100000:2.0:1789762802.501276:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325789647438 time 325789765561 state 0x0 ref 1 00000100:00100000:2.0:1789762802.501292:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325839765561 time 325789765561 state 0x1 ref 1 00000100:00100000:1.0:1789762802.535989:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325839765561 time 325839886168 state 0x0 ref 1 00000100:00100000:1.0:1789762802.536011:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014988288:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762802.536061:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 44040192, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x15007 00010000:00010000:1.0:1789762802.536070:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d95 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.536118:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:1.0:1789762802.536124:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.536132:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.536148:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762802.547009:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967852 is committed 00002000:00010000:1.0:1789762802.547078:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 44040192 -> 45088768 00002000:00010000:1.0:1789762802.547100:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 86023 -> 88071 00010000:00010000:1.0:1789762802.547111:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d95 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.547122:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d95 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.547129:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.547134:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.547138:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d95 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.547147:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d95 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.547163:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d95 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.547168:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d95 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.547235:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014988288:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11228us (46086us total) trans 4294967852 rc 0/0 00000100:00100000:1.0:1789762802.547245:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 535 00000100:00100000:1.0:1789762802.547255:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325839765561 time 325839886168 state 0x2 ref 0 00000100:00100000:0.0:1789762802.549209:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.549288:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014988416 00000100:00100000:2.0:1789762802.549297:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014988416 00000100:00100000:2.0:1789762802.549355:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325839765561 time 325839886168 state 0x0 ref 1 00000100:00100000:2.0:1789762802.549369:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325889886168 time 325839886168 state 0x1 ref 1 00000100:00100000:1.0:1789762802.586113:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325889886168 time 325889987464 state 0x0 ref 1 00000100:00100000:1.0:1789762802.586137:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014988416:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762802.586190:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 45088768, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x15807 00010000:00010000:1.0:1789762802.586198:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d9c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.586213:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:1.0:1789762802.586217:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.586222:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.586235:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762802.597182:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967853 is committed 00002000:00010000:1.0:1789762802.597192:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 45088768 -> 46137344 00002000:00010000:1.0:1789762802.597199:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 88071 -> 90119 00010000:00010000:1.0:1789762802.597211:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d9c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.597221:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d9c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.597234:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.597240:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.597244:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d9c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.597251:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d9c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.597267:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d9c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.597272:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03d9c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.597352:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014988416:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11208us (48131us total) trans 4294967853 rc 0/0 00000100:00100000:1.0:1789762802.597362:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 536 00000100:00100000:1.0:1789762802.597373:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325889886168 time 325889987464 state 0x2 ref 0 00000100:00100000:3.0:1789762802.599469:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.599520:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014988544 00000100:00100000:2.0:1789762802.599530:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014988544 00000100:00100000:2.0:1789762802.599574:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325889886168 time 325889987464 state 0x0 ref 1 00000100:00100000:2.0:1789762802.599587:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325939987464 time 325889987464 state 0x1 ref 1 00010000:00010000:1.0:1789762802.615202:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762802.636167:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325939987464 time 325940065778 state 0x0 ref 1 00000100:00100000:1.0:1789762802.636189:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014988544:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762802.636243:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 46137344, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x16007 00010000:00010000:1.0:1789762802.636251:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03da3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.636265:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:1.0:1789762802.636270:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.636275:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03da3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.636287:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03da3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762802.647767:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967854 is committed 00002000:00010000:1.0:1789762802.647784:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 46137344 -> 47185920 00002000:00010000:1.0:1789762802.647789:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 90119 -> 92167 00010000:00010000:1.0:1789762802.647802:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03da3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.647813:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03da3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.647821:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03da3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.647826:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.647831:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03da3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.647844:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03da3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.647862:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03da3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.647867:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03da3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.647958:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014988544:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11771us (48486us total) trans 4294967854 rc 0/0 00000100:00100000:1.0:1789762802.647969:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 537 00000100:00100000:1.0:1789762802.647980:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325939987464 time 325940065778 state 0x2 ref 0 00000100:00100000:3.0:1789762802.650060:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.650122:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014988672 00000100:00100000:2.0:1789762802.650132:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014988672 00000100:00100000:2.0:1789762802.650184:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325939987464 time 325940065778 state 0x0 ref 1 00000100:00100000:2.0:1789762802.650209:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325990065778 time 325940065778 state 0x1 ref 1 00000100:00100000:1.0:1789762802.686301:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325990065778 time 325990191682 state 0x0 ref 1 00000100:00100000:1.0:1789762802.686335:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b38700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014988672:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762802.686421:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 47185920, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x16807 00010000:00010000:1.0:1789762802.686434:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03daa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.686456:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=325s v=(0 0) 00010000:00010000:1.0:1789762802.686462:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.686468:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03daa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.686483:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03daa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762802.698586:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967855 is committed 00002000:00010000:1.0:1789762802.698654:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 47185920 -> 48234496 00002000:00010000:1.0:1789762802.698659:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 92167 -> 94215 00010000:00010000:1.0:1789762802.698671:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03daa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.698682:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03daa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.698689:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03daa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.698695:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.698699:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03daa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.698706:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03daa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.698725:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03daa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.698731:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03daa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.698809:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b38700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014988672:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12486us (48748us total) trans 4294967855 rc 0/0 00000100:00100000:1.0:1789762802.698819:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 538 00000100:00100000:1.0:1789762802.698830:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325990065778 time 325990191682 state 0x2 ref 0 00000100:00100000:2.0:1789762802.701005:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762802.701051:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014988800 00000100:00100000:1.0:1789762802.701060:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014988800 00000100:00100000:1.0:1789762802.701126:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 325990065778 time 325990191682 state 0x0 ref 1 00000100:00100000:1.0:1789762802.701139:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326040191682 time 325990191682 state 0x1 ref 1 00000100:00100000:1.0:1789762802.736384:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326040191682 time 326040277493 state 0x0 ref 1 00000100:00100000:1.0:1789762802.736414:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014988800:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762802.736489:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 48234496, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x17007 00010000:00010000:1.0:1789762802.736498:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.736515:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762802.736519:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.736525:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.736539:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762802.748596:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967856 is committed 00002000:00010000:1.0:1789762802.748666:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 48234496 -> 49283072 00002000:00010000:1.0:1789762802.748671:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 94215 -> 96265 00010000:00010000:1.0:1789762802.748684:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.748695:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.748702:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.748707:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.748711:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.748722:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.748746:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.748755:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.748846:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014988800:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12442us (47840us total) trans 4294967856 rc 0/0 00000100:00100000:1.0:1789762802.748860:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 539 00000100:00100000:1.0:1789762802.748874:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326040191682 time 326040277493 state 0x2 ref 0 00000100:00100000:3.0:1789762802.751137:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.751181:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014988928 00000100:00100000:2.0:1789762802.751191:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014988928 00000100:00100000:2.0:1789762802.751239:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326040191682 time 326040277493 state 0x0 ref 1 00000100:00100000:2.0:1789762802.751253:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326090277493 time 326040277493 state 0x1 ref 1 00000100:00100000:0.0:1789762802.786306:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762802.786381:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.786411:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014989184 00000100:00100000:0.0:1789762802.786418:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762802.786425:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014989184 00000100:00100000:3.0:1789762802.786444:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014989056 00000100:00100000:1.0:1789762802.786450:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326090277493 time 326090347669 state 0x0 ref 1 00000100:00100000:3.0:1789762802.786452:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014989056 00000100:00100000:0.0:1789762802.786462:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762802.786478:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b39c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014988928:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762802.786495:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014989312 00000100:00100000:0.0:1789762802.786501:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014989312 00000100:00100000:2.0:1789762802.786510:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014989440 00000100:00100000:2.0:1789762802.786513:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014989440 00000100:00100000:3.0:1789762802.786519:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 300 00000100:00100000:3.0:1789762802.786534:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85085050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014989056:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762802.786548:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 119 00000100:00100000:2.0:1789762802.786548:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 238 00002000:00010000:1.0:1789762802.786549:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 49283072, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x17809 00010000:00010000:1.0:1789762802.786559:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762802.786561:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2365:x1876702014989312:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762802.786561:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702014989184:12345-192.168.203.48@tcp:400:kworker.0 00010000:00010000:1.0:1789762802.786585:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762802.786591:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.786602:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.786623:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762802.786635:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85085050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702014989056:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 99us (246us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762802.786636:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2365:x1876702014989312:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 72us (209us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762802.786636:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702014989184:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 72us (334us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762802.786656:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 119 00000100:00100000:2.0:1789762802.786662:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 238 00000100:00100000:3.0:1789762802.786663:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 300 00000100:00100000:2.0:1789762802.786684:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 239 00000100:00100000:2.0:1789762802.786695:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702014989440:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762802.786725:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702014989440:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 33us (269us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762802.786729:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762802.786733:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 239 00000100:00080000:0.0:1789762802.786735:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:3.0:1789762802.799057:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967857 is committed 00002000:00010000:1.0:1789762802.799084:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 49283072 -> 50331648 00002000:00010000:1.0:1789762802.799112:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 96265 -> 98313 00010000:00010000:1.0:1789762802.799127:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.799139:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.799156:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.799166:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.799172:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.799182:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.799204:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.799213:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03db8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.799292:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b39c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014988928:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12817us (48151us total) trans 4294967857 rc 0/0 00000100:00100000:1.0:1789762802.799305:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 540 00000100:00100000:1.0:1789762802.799319:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326090277493 time 326090347669 state 0x2 ref 0 00000100:00100000:3.0:1789762802.801612:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762802.801679:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014989568 00000100:00100000:1.0:1789762802.801693:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014989568 00000100:00100000:1.0:1789762802.801764:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326090277493 time 326090347669 state 0x0 ref 1 00000100:00100000:1.0:1789762802.801789:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326140347669 time 326090347669 state 0x1 ref 1 00000004:00080000:1.0:1789762802.807252:0:6868:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 326111161981ns, next at 327111158991ns, rc = 0 00000100:00100000:2.0:1789762802.807273:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc555180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022165504:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762802.807365:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762802.807421:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022165504 00000100:00100000:0.0:1789762802.807428:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022165504 00000100:00100000:0.0:1789762802.807480:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 101 00000100:00100000:0.0:1789762802.807490:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022165504:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762802.807512:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3803185152 avail 3798990848 00000100:00100000:0.0:1789762802.807561:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022165504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762802.807570:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 101 00000100:00100000:2.0:1789762802.807591:0:3274:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967827 gen 2 00000004:00080000:2.0:1789762802.807597:0:3274:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981e98e7b480, transno 4294967827 00000100:00100000:2.0:1789762802.807621:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc555180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022165504:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762802.836528:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326140347669 time 326140420817 state 0x0 ref 1 00000100:00100000:1.0:1789762802.836553:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b39f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014989568:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762802.836613:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 50331648, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x18009 00010000:00010000:1.0:1789762802.836622:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dbf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.836639:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762802.836643:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.836649:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.836666:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762802.849489:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967858 is committed 00002000:00010000:1.0:1789762802.849511:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 50331648 -> 51380224 00002000:00010000:1.0:1789762802.849518:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 98313 -> 100361 00010000:00010000:1.0:1789762802.849532:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dbf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.849542:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dbf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.849551:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.849557:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.849561:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dbf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.849568:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dbf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.849587:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dbf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.849592:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dbf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.849670:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b39f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014989568:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13123us (48061us total) trans 4294967858 rc 0/0 00000100:00100000:1.0:1789762802.849680:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 541 00000100:00100000:1.0:1789762802.849690:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326140347669 time 326140420817 state 0x2 ref 0 00000100:00100000:3.0:1789762802.851899:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762802.851937:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014989696 00000100:00100000:1.0:1789762802.851947:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014989696 00000100:00100000:1.0:1789762802.851994:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326140347669 time 326140420817 state 0x0 ref 1 00000100:00100000:1.0:1789762802.852011:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326190420817 time 326140420817 state 0x1 ref 1 00000100:00100000:1.0:1789762802.886613:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326190420817 time 326190503589 state 0x0 ref 1 00000100:00100000:1.0:1789762802.886645:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b39880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014989696:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762802.886730:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 51380224, mtime: 0x6aad9cf2, atime: 0x0, ctime: 0x6aad9cf2, blocks: 0x18809 00010000:00010000:1.0:1789762802.886742:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dc6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.886763:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762802.886769:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.886777:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.886801:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762802.898665:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967859 is committed 00002000:00010000:1.0:1789762802.898681:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 51380224 -> 52428800 00002000:00010000:1.0:1789762802.898688:0:11723:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb mtime from disk: 1789762802 -> 1789762803 00002000:00010000:1.0:1789762802.898692:0:11723:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb ctime from disk: 1789762802 -> 1789762803 00002000:00010000:1.0:1789762802.898696:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 100361 -> 102409 00010000:00010000:1.0:1789762802.898712:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dc6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.898730:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dc6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.898742:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.898752:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.898759:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dc6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.898771:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dc6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.898795:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dc6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.898801:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dc6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.898883:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b39880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014989696:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12249us (46984us total) trans 4294967859 rc 0/0 00000100:00100000:1.0:1789762802.898892:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 542 00000100:00100000:1.0:1789762802.898903:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326190420817 time 326190503589 state 0x2 ref 0 00000100:00100000:2.0:1789762802.901131:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762802.901178:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014989824 00000100:00100000:0.0:1789762802.901187:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014989824 00000100:00100000:0.0:1789762802.901235:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326190420817 time 326190503589 state 0x0 ref 1 00000100:00100000:0.0:1789762802.901256:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326240503589 time 326190503589 state 0x1 ref 1 00000100:00100000:1.0:1789762802.936716:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326240503589 time 326240604537 state 0x0 ref 1 00000100:00100000:1.0:1789762802.936752:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014989824:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762802.936821:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 52428800, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x19009 00010000:00010000:1.0:1789762802.936830:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dcd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.936847:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762802.936852:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.936858:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.936871:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762802.948778:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967860 is committed 00002000:00010000:1.0:1789762802.948789:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 52428800 -> 53477376 00002000:00010000:1.0:1789762802.948794:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 102409 -> 104457 00010000:00010000:1.0:1789762802.948809:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dcd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.948820:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dcd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.948828:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.948833:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762802.948838:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dcd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.948846:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dcd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.948864:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dcd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.948869:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dcd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762802.948952:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014989824:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12213us (47817us total) trans 4294967860 rc 0/0 00000100:00100000:1.0:1789762802.948965:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 543 00000100:00100000:1.0:1789762802.948977:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326240503589 time 326240604537 state 0x2 ref 0 00000100:00100000:3.0:1789762802.951160:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762802.951196:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014989952 00000100:00100000:1.0:1789762802.951205:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014989952 00000100:00100000:1.0:1789762802.951253:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326240503589 time 326240604537 state 0x0 ref 1 00000100:00100000:1.0:1789762802.951267:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326290604537 time 326240604537 state 0x1 ref 1 00000100:00100000:1.0:1789762802.986791:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326290604537 time 326290683598 state 0x0 ref 1 00000100:00100000:1.0:1789762802.986818:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b38380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014989952:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762802.986881:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 53477376, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x19809 00010000:00010000:1.0:1789762802.986890:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dd4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.986907:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762802.986911:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762802.986916:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762802.986930:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762802.999332:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967861 is committed 00002000:00010000:2.0:1789762802.999333:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:2.0:1789762802.999353:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 104457 -> 106505 00010000:00010000:2.0:1789762802.999368:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dd4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.999383:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dd4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.999395:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.999405:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762802.999411:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dd4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.999422:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dd4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.999443:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dd4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762802.999449:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dd4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762802.999539:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b38380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014989952:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12729us (48379us total) trans 4294967861 rc 0/0 00000100:00100000:2.0:1789762802.999549:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 544 00000100:00100000:2.0:1789762802.999560:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326290604537 time 326290683598 state 0x2 ref 0 00000100:00100000:3.0:1789762803.001741:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.001790:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014990080 00000100:00100000:1.0:1789762803.001800:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014990080 00000100:00080000:0.0:1789762803.001855:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762803.001865:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326290604537 time 326290683598 state 0x0 ref 1 00000100:00100000:1.0:1789762803.001879:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326340683598 time 326290683598 state 0x1 ref 1 00000100:00100000:1.0:1789762803.036853:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326340683598 time 326340744152 state 0x0 ref 1 00000100:00100000:1.0:1789762803.036882:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990080:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.036960:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 54525952, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1a009 00010000:00010000:1.0:1789762803.036970:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ddb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.036992:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.036997:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.037003:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ddb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.037018:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ddb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762803.050052:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967862 is committed 00002000:00010000:1.0:1789762803.050081:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 54525952 -> 55574528 00002000:00010000:1.0:1789762803.050103:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 106505 -> 108553 00010000:00010000:1.0:1789762803.050117:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ddb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.050128:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ddb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.050135:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ddb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.050141:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.050145:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ddb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.050154:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ddb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.050178:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ddb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.050184:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ddb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.050263:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990080:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13388us (48520us total) trans 4294967862 rc 0/0 00000100:00100000:1.0:1789762803.050271:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 545 00000100:00100000:1.0:1789762803.050283:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326340683598 time 326340744152 state 0x2 ref 0 00000100:00100000:3.0:1789762803.052544:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762803.052591:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014990208 00000100:00100000:0.0:1789762803.052601:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014990208 00000100:00100000:0.0:1789762803.052651:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326340683598 time 326340744152 state 0x0 ref 1 00000100:00100000:0.0:1789762803.052672:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326390744152 time 326340744152 state 0x1 ref 1 00000100:00100000:1.0:1789762803.086928:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326390744152 time 326390819641 state 0x0 ref 1 00000100:00100000:1.0:1789762803.086952:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990208:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.087008:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 55574528, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1a809 00010000:00010000:1.0:1789762803.087016:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03de2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.087037:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.087043:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.087052:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03de2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.087070:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03de2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762803.101279:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967863 is committed 00002000:00010000:1.0:1789762803.101307:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 55574528 -> 56623104 00002000:00010000:1.0:1789762803.101315:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 108553 -> 110601 00010000:00010000:1.0:1789762803.101327:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03de2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.101352:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03de2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.101360:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03de2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.101365:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.101369:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03de2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.101375:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03de2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.101390:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03de2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.101396:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03de2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.101480:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990208:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14529us (48934us total) trans 4294967863 rc 0/0 00000100:00100000:1.0:1789762803.101489:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 546 00000100:00100000:1.0:1789762803.101500:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326390744152 time 326390819641 state 0x2 ref 0 00000100:00100000:3.0:1789762803.103856:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.103932:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014990336 00000100:00100000:1.0:1789762803.103941:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014990336 00000100:00100000:1.0:1789762803.103984:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326390744152 time 326390819641 state 0x0 ref 1 00000100:00100000:1.0:1789762803.103995:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326440819641 time 326390819641 state 0x1 ref 1 00000100:00100000:2.0:1789762803.137029:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326440819641 time 326440923665 state 0x0 ref 1 00000100:00100000:2.0:1789762803.137058:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990336:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762803.137148:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 56623104, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1b009 00010000:00010000:2.0:1789762803.137157:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03de9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.137176:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:2.0:1789762803.137183:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762803.137192:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03de9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762803.137212:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03de9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762803.152160:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967864 is committed 00002000:00010000:1.0:1789762803.152177:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:1.0:1789762803.152184:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 110601 -> 112649 00010000:00010000:1.0:1789762803.152196:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03de9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.152207:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03de9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.152214:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03de9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.152219:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.152223:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03de9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.152230:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03de9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.152246:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03de9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.152252:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03de9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.152332:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990336:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15282us (48478us total) trans 4294967864 rc 0/0 00000100:00100000:1.0:1789762803.152353:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 547 00000100:00100000:1.0:1789762803.152364:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326440819641 time 326440923665 state 0x2 ref 0 00000100:00100000:2.0:1789762803.154641:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.154717:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014990464 00000100:00100000:1.0:1789762803.154725:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014990464 00000100:00100000:1.0:1789762803.154764:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326440819641 time 326440923665 state 0x0 ref 1 00000100:00100000:1.0:1789762803.154776:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326490923665 time 326440923665 state 0x1 ref 1 00000100:00100000:1.0:1789762803.187084:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326490923665 time 326490984428 state 0x0 ref 1 00000100:00100000:1.0:1789762803.187140:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014990464:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.187190:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 57671680, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1b809 00010000:00010000:1.0:1789762803.187197:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.187214:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.187218:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.187223:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.187233:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762803.205269:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967865 is committed 00002000:00010000:1.0:1789762803.205284:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:1.0:1789762803.205290:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 112649 -> 114697 00010000:00010000:1.0:1789762803.205301:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.205311:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.205318:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.205323:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.205326:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.205333:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.205367:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.205373:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.205435:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014990464:12345-192.168.203.48@tcp:4:dd.500 Request processed in 18305us (50798us total) trans 4294967865 rc 0/0 00000100:00100000:1.0:1789762803.205445:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 548 00000100:00100000:1.0:1789762803.205453:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326490923665 time 326490984428 state 0x2 ref 0 00000100:00100000:0.0:1789762803.207563:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762803.207617:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014990592 00000100:00100000:2.0:1789762803.207630:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014990592 00000100:00100000:2.0:1789762803.207687:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326490923665 time 326490984428 state 0x0 ref 1 00000100:00100000:2.0:1789762803.207701:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326540984428 time 326490984428 state 0x1 ref 1 00000100:00100000:1.0:1789762803.237214:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326540984428 time 326541109460 state 0x0 ref 1 00000100:00100000:1.0:1789762803.237241:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990592:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.237298:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 58720256, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1c009 00010000:00010000:1.0:1789762803.237306:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.237319:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.237323:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.237328:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.237356:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762803.251685:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967866 is committed 00002000:00010000:1.0:1789762803.251742:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:1.0:1789762803.251747:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 114697 -> 116745 00010000:00010000:1.0:1789762803.251757:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.251767:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.251774:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.251779:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.251782:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.251789:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.251804:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.251808:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03df7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.251888:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990592:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14656us (44324us total) trans 4294967866 rc 0/0 00000100:00100000:1.0:1789762803.251898:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 549 00000100:00100000:1.0:1789762803.251908:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326540984428 time 326541109460 state 0x2 ref 0 00000100:00100000:0.0:1789762803.253980:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.254018:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014990720 00000100:00100000:1.0:1789762803.254027:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014990720 00000100:00100000:1.0:1789762803.254070:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326540984428 time 326541109460 state 0x0 ref 1 00000100:00100000:1.0:1789762803.254108:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326591109460 time 326541109460 state 0x1 ref 1 00000100:00100000:1.0:1789762803.287271:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326591109460 time 326591167894 state 0x0 ref 1 00000100:00100000:1.0:1789762803.287293:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014990720:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.287355:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 59768832, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1c809 00010000:00010000:1.0:1789762803.287364:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dfe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.287377:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.287381:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.287386:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.287397:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762803.301125:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967867 is committed 00002000:00010000:1.0:1789762803.301130:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 59768832 -> 60817408 00002000:00010000:1.0:1789762803.301136:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 116745 -> 118793 00010000:00010000:1.0:1789762803.301146:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dfe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.301155:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dfe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.301161:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.301166:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.301170:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dfe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.301177:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dfe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.301190:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dfe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.301195:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03dfe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.301252:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014990720:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13964us (47271us total) trans 4294967867 rc 0/0 00000100:00100000:1.0:1789762803.301266:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 550 00000100:00100000:1.0:1789762803.301277:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326591109460 time 326591167894 state 0x2 ref 0 00000100:00100000:0.0:1789762803.303266:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.303305:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014990848 00000100:00100000:1.0:1789762803.303313:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014990848 00000100:00100000:1.0:1789762803.303368:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326591109460 time 326591167894 state 0x0 ref 1 00000100:00100000:1.0:1789762803.303384:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326641167894 time 326591167894 state 0x1 ref 1 00000004:00080000:1.0:1789762803.319288:0:7726:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 326623199459ns, next at 327623192251ns, rc = 0 00000100:00100000:3.0:1789762803.319350:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc557100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022165632:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762803.319351:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc585880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022165760:0@lo:400:kworker.0 00000100:00100000:1.0:1789762803.319364:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfedd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022165888:0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.319386:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfede300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022166016:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.319442:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762803.319475:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762803.319486:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762803.319516:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762803.319552:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022166272:0@lo:400:kworker.0 00000100:00100000:3.0:1789762803.319570:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfedd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022166144:0@lo:400:kworker.0 00000100:00100000:1.0:1789762803.319590:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022165888 00000100:00100000:0.0:1789762803.319592:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762803.319597:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022165888 00000100:00100000:2.0:1789762803.319608:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022166016 00000100:00100000:2.0:1789762803.319617:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022166016 00000100:00100000:3.0:1789762803.319627:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762803.319657:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022165632 00000100:00100000:1.0:1789762803.319657:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022166144 00000100:00100000:1.0:1789762803.319664:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022166144 00000100:00100000:0.0:1789762803.319666:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022165632 00000100:00100000:1.0:1789762803.319730:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:2.0:1789762803.319746:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:1.0:1789762803.319748:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022165888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.319757:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 102 00000100:00100000:3.0:1789762803.319767:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:2.0:1789762803.319770:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022166016:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.319778:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022165632:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762803.319794:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022166144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762803.319809:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022165888:12345-0@lo:400:kworker.0 Request processed in 69us (338us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762803.319816:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00002000:00000024:0.0:1789762803.319835:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 98762752 pending 27262976 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762803.319868:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022165760 00000100:00100000:2.0:1789762803.319879:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022166016:12345-0@lo:400:kworker.0 Request processed in 109us (360us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762803.319881:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022165760 00000100:00100000:3.0:1789762803.319895:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022166144:12345-0@lo:400:kworker.0 Request processed in 104us (267us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762803.319909:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00100000:0.0:1789762803.319911:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022165632:12345-0@lo:13:osp-pre-1-0.0 Request processed in 128us (414us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762803.319921:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:0.0:1789762803.319928:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 102 00000100:00100000:1.0:1789762803.319929:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 301 00000100:00100000:1.0:1789762803.319950:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85087850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022165760:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762803.319952:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762803.319964:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762803.320015:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022166272 00000100:00100000:3.0:1789762803.320025:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022166272 00000100:00100000:1.0:1789762803.320026:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85087850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022165760:12345-0@lo:400:kworker.0 Request processed in 84us (582us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762803.320036:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 301 00000100:00100000:2.0:1789762803.320046:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfede300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022166016:0@lo:400:kworker.0 00000100:00100000:0.0:1789762803.320055:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc585880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022165760:0@lo:400:kworker.0 00000100:00100000:3.0:1789762803.320143:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 120 00000100:00100000:1.0:1789762803.320160:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfedd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022165888:0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.320165:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfedd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022166144:0@lo:400:kworker.0 00000100:00100000:3.0:1789762803.320167:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022166272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762803.320211:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc557100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022165632:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762803.320248:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022166272:12345-0@lo:400:kworker.0 Request processed in 92us (655us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762803.320257:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 120 00000100:00100000:0.0:1789762803.320295:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022166272:0@lo:400:kworker.0 00000100:00100000:1.0:1789762803.337326:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326641167894 time 326641222814 state 0x0 ref 1 00000100:00100000:1.0:1789762803.337360:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990848:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.337408:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 60817408, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1d009 00010000:00010000:1.0:1789762803.337415:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e05 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.337428:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.337432:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.337437:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.337448:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762803.351619:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967868 is committed 00002000:00010000:1.0:1789762803.351648:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 60817408 -> 61865984 00002000:00010000:1.0:1789762803.351655:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 118793 -> 120841 00010000:00010000:1.0:1789762803.351668:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e05 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.351677:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e05 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.351685:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.351690:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.351693:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e05 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.351700:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e05 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.351714:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e05 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.351720:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e05 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.351791:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990848:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14439us (48524us total) trans 4294967868 rc 0/0 00000100:00100000:1.0:1789762803.351799:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 551 00000100:00100000:1.0:1789762803.351808:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326641167894 time 326641222814 state 0x2 ref 0 00000100:00100000:3.0:1789762803.353882:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762803.353932:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014990976 00000100:00100000:2.0:1789762803.353942:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014990976 00000100:00100000:2.0:1789762803.354006:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326641167894 time 326641222814 state 0x0 ref 1 00000100:00100000:2.0:1789762803.354024:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326691222814 time 326641222814 state 0x1 ref 1 00000100:00100000:1.0:1789762803.387415:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326691222814 time 326691313301 state 0x0 ref 1 00000100:00100000:1.0:1789762803.387433:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990976:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.387475:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 61865984, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1d809 00010000:00010000:1.0:1789762803.387482:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e0c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.387494:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.387498:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.387502:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.387512:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762803.401347:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967869 is committed 00002000:00010000:1.0:1789762803.401358:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 61865984 -> 62914560 00002000:00010000:1.0:1789762803.401363:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 120841 -> 122889 00010000:00010000:1.0:1789762803.401374:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e0c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.401383:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e0c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.401389:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.401394:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.401398:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e0c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.401409:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e0c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.401421:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e0c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.401426:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e0c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.401480:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014990976:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14051us (47603us total) trans 4294967869 rc 0/0 00000100:00100000:1.0:1789762803.401488:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 552 00000100:00100000:1.0:1789762803.401495:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326691222814 time 326691313301 state 0x2 ref 0 00000100:00100000:3.0:1789762803.403664:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.403704:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014991104 00000100:00100000:1.0:1789762803.403711:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014991104 00000100:00100000:1.0:1789762803.403755:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326691222814 time 326691313301 state 0x0 ref 1 00000100:00100000:1.0:1789762803.403768:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326741313301 time 326691313301 state 0x1 ref 1 00000100:00100000:1.0:1789762803.437593:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326741313301 time 326741488612 state 0x0 ref 1 00000100:00100000:1.0:1789762803.437618:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014991104:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.437670:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 62914560, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1e009 00010000:00010000:1.0:1789762803.437677:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e13 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.437690:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.437693:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.437698:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.437709:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762803.451984:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:1.0:1789762803.451991:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 122889 -> 124937 00000001:00080000:3.0:1789762803.451992:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967870 is committed 00010000:00010000:1.0:1789762803.452002:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e13 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452013:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e13 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452020:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452025:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.452029:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e13 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452040:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e13 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452053:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e13 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.452058:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e13 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.452147:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014991104:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14536us (48485us total) trans 4294967870 rc 0/0 00000100:00100000:1.0:1789762803.452155:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 553 00000100:00100000:1.0:1789762803.452164:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326741313301 time 326741488612 state 0x2 ref 0 00000100:00100000:2.0:1789762803.454367:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.454407:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014991232 00000100:00100000:1.0:1789762803.454416:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014991232 00000100:00100000:1.0:1789762803.454452:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326741313301 time 326741488612 state 0x0 ref 1 00000100:00100000:1.0:1789762803.454463:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326791488612 time 326741488612 state 0x1 ref 1 00000100:00100000:1.0:1789762803.487676:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326791488612 time 326791567099 state 0x0 ref 1 00000100:00100000:1.0:1789762803.487708:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014991232:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.487766:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 63963136, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1e809 00010000:00010000:1.0:1789762803.487777:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e1a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.487795:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.487801:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.487818:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.487835:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762803.502048:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967871 is committed 00002000:00010000:1.0:1789762803.502076:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 63963136 -> 65011712 00002000:00010000:1.0:1789762803.502083:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 124937 -> 126985 00010000:00010000:1.0:1789762803.502121:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e1a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.502135:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.502146:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.502155:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.502160:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.502170:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.502190:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.502200:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.502281:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014991232:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14584us (47914us total) trans 4294967871 rc 0/0 00000100:00100000:1.0:1789762803.502293:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 554 00000100:00100000:1.0:1789762803.502307:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326791488612 time 326791567099 state 0x2 ref 0 00000100:00100000:3.0:1789762803.504443:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762803.504491:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014991360 00000100:00100000:2.0:1789762803.504503:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014991360 00000100:00100000:2.0:1789762803.504563:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326791488612 time 326791567099 state 0x0 ref 1 00000100:00100000:2.0:1789762803.504577:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326841567099 time 326791567099 state 0x1 ref 1 00000100:00100000:1.0:1789762803.537789:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326841567099 time 326841685687 state 0x0 ref 1 00000100:00100000:1.0:1789762803.537815:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014991360:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.537864:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 65011712, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1f009 00010000:00010000:1.0:1789762803.537871:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e21 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.537884:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.537888:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.537893:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.537903:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762803.551910:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967872 is committed 00002000:00010000:1.0:1789762803.551960:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 65011712 -> 66060288 00002000:00010000:1.0:1789762803.551964:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 126985 -> 129033 00010000:00010000:1.0:1789762803.551973:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e21 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.551981:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e21 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.551987:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.551992:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.551996:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e21 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.552003:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e21 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.552017:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e21 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.552021:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e21 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.552121:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014991360:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14281us (47649us total) trans 4294967872 rc 0/0 00000100:00100000:1.0:1789762803.552130:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 555 00000100:00100000:1.0:1789762803.552139:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326841567099 time 326841685687 state 0x2 ref 0 00000100:00100000:3.0:1789762803.554154:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.554202:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014991488 00000100:00100000:1.0:1789762803.554211:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014991488 00000100:00100000:1.0:1789762803.554247:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326841567099 time 326841685687 state 0x0 ref 1 00000100:00100000:1.0:1789762803.554258:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326891685687 time 326841685687 state 0x1 ref 1 00000100:00100000:1.0:1789762803.587851:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326891685687 time 326891752558 state 0x0 ref 1 00000100:00100000:1.0:1789762803.587870:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014991488:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.587906:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 66060288, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x1f809 00010000:00010000:1.0:1789762803.587913:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.587924:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.587928:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.587932:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.587942:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762803.602949:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967873 is committed 00002000:00010000:1.0:1789762803.602954:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 66060288 -> 67108864 00002000:00010000:1.0:1789762803.602975:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 129033 -> 131081 00010000:00010000:1.0:1789762803.602986:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.602996:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e28 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.603003:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.603009:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.603012:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.603018:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e28 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.603032:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e28 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.603037:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e28 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.603131:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014991488:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15265us (48981us total) trans 4294967873 rc 0/0 00000100:00100000:1.0:1789762803.603141:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 556 00000100:00100000:1.0:1789762803.603151:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326891685687 time 326891752558 state 0x2 ref 0 00000100:00100000:0.0:1789762803.605251:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.605284:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014991616 00000100:00100000:1.0:1789762803.605293:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014991616 00000100:00100000:1.0:1789762803.605328:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326891685687 time 326891752558 state 0x0 ref 1 00000100:00100000:1.0:1789762803.605351:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326941752558 time 326891752558 state 0x1 ref 1 00000100:00100000:1.0:1789762803.637958:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326941752558 time 326941854470 state 0x0 ref 1 00000100:00100000:1.0:1789762803.637981:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014991616:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.638027:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 67108864, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x20009 00010000:00010000:1.0:1789762803.638034:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.638048:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.638052:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.638056:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.638067:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.639230:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789762803.651694:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967874 is committed 00002000:00010000:1.0:1789762803.651735:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 67108864 -> 68157440 00002000:00010000:1.0:1789762803.651740:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 131081 -> 133129 00010000:00010000:1.0:1789762803.651750:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.651759:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.651765:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.651771:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.651774:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.651781:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.651794:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.651799:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.651863:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014991616:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13890us (46613us total) trans 4294967874 rc 0/0 00000100:00100000:1.0:1789762803.651871:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 557 00000100:00100000:1.0:1789762803.651880:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326941752558 time 326941854470 state 0x2 ref 0 00000100:00100000:0.0:1789762803.654011:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762803.654067:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014991744 00000100:00100000:2.0:1789762803.654078:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014991744 00000100:00100000:2.0:1789762803.654154:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326941752558 time 326941854470 state 0x0 ref 1 00000100:00100000:2.0:1789762803.654168:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326991854470 time 326941854470 state 0x1 ref 1 00000100:00100000:1.0:1789762803.688072:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326991854470 time 326991966076 state 0x0 ref 1 00000100:00100000:1.0:1789762803.688132:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014991744:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.688183:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 68157440, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x20809 00010000:00010000:1.0:1789762803.688193:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.688210:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=326s v=(0 0) 00010000:00010000:1.0:1789762803.688215:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.688219:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.688231:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762803.702205:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967875 is committed 00002000:00010000:1.0:1789762803.702251:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 68157440 -> 69206016 00002000:00010000:1.0:1789762803.702256:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 133129 -> 135177 00010000:00010000:1.0:1789762803.702265:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.702275:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.702282:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.702287:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.702291:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.702297:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.702312:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.702318:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.702398:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014991744:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14308us (48387us total) trans 4294967875 rc 0/0 00000100:00100000:1.0:1789762803.702407:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 558 00000100:00100000:1.0:1789762803.702416:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326991854470 time 326991966076 state 0x2 ref 0 00000100:00100000:2.0:1789762803.704573:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.704647:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014991872 00000100:00100000:1.0:1789762803.704656:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014991872 00000100:00100000:1.0:1789762803.704694:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 326991854470 time 326991966076 state 0x0 ref 1 00000100:00100000:1.0:1789762803.704705:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327041966076 time 326991966076 state 0x1 ref 1 00000100:00100000:1.0:1789762803.738149:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327041966076 time 327042046639 state 0x0 ref 1 00000100:00100000:1.0:1789762803.738172:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014991872:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.738217:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 69206016, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x21009 00010000:00010000:1.0:1789762803.738226:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.738243:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762803.738249:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.738254:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.738269:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762803.753248:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967876 is committed 00002000:00010000:1.0:1789762803.753291:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 69206016 -> 70254592 00002000:00010000:1.0:1789762803.753296:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 135177 -> 137225 00010000:00010000:1.0:1789762803.753305:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.753314:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.753322:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.753327:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.753344:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.753353:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.753371:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.753377:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.753449:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014991872:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15284us (48881us total) trans 4294967876 rc 0/0 00000100:00100000:1.0:1789762803.753462:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 559 00000100:00100000:1.0:1789762803.753470:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327041966076 time 327042046639 state 0x2 ref 0 00000100:00100000:2.0:1789762803.755562:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.755600:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014992000 00000100:00100000:1.0:1789762803.755608:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014992000 00000100:00100000:1.0:1789762803.755644:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327041966076 time 327042046639 state 0x0 ref 1 00000100:00100000:1.0:1789762803.755658:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327092046639 time 327042046639 state 0x1 ref 1 00000100:00100000:1.0:1789762803.788199:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327092046639 time 327092100558 state 0x0 ref 1 00000100:00100000:1.0:1789762803.788215:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014992000:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.788250:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 70254592, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x21809 00010000:00010000:1.0:1789762803.788257:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.788268:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762803.788272:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.788276:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.788286:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762803.801795:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967877 is committed 00002000:00010000:1.0:1789762803.801820:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 70254592 -> 71303168 00002000:00010000:1.0:1789762803.801826:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 137225 -> 139273 00010000:00010000:1.0:1789762803.801838:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.801849:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.801860:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.801865:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.801869:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.801875:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.801887:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.801893:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.801952:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014992000:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13739us (46393us total) trans 4294967877 rc 0/0 00000100:00100000:1.0:1789762803.801959:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 560 00000100:00100000:1.0:1789762803.801968:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327092046639 time 327092100558 state 0x2 ref 0 00000100:00100000:2.0:1789762803.804748:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762803.804814:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014992128 00000100:00100000:0.0:1789762803.804827:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014992128 00000100:00100000:0.0:1789762803.804878:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327092046639 time 327092100558 state 0x0 ref 1 00000100:00100000:0.0:1789762803.804891:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327142100558 time 327092100558 state 0x1 ref 1 00000100:00100000:1.0:1789762803.838321:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327142100558 time 327142215981 state 0x0 ref 1 00000100:00100000:1.0:1789762803.838357:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014992128:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.838421:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 71303168, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x22009 00010000:00010000:1.0:1789762803.838432:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.838449:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762803.838454:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.838459:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.838471:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762803.853564:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967878 is committed 00002000:00010000:1.0:1789762803.853597:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 71303168 -> 72351744 00002000:00010000:1.0:1789762803.853602:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 139273 -> 141321 00010000:00010000:1.0:1789762803.853611:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.853620:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.853627:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.853632:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.853635:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.853646:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.853661:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.853666:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.853748:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014992128:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15398us (49011us total) trans 4294967878 rc 0/0 00000100:00100000:1.0:1789762803.853755:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 561 00000100:00100000:1.0:1789762803.853765:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327142100558 time 327142215981 state 0x2 ref 0 00000100:00100000:3.0:1789762803.856187:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.856232:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014992256 00000100:00100000:1.0:1789762803.856240:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014992256 00000100:00100000:1.0:1789762803.856290:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327142100558 time 327142215981 state 0x0 ref 1 00000100:00100000:1.0:1789762803.856307:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327192215981 time 327142215981 state 0x1 ref 1 00000100:00100000:1.0:1789762803.888424:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327192215981 time 327192316585 state 0x0 ref 1 00000100:00100000:1.0:1789762803.888451:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014992256:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.888500:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 72351744, mtime: 0x6aad9cf3, atime: 0x0, ctime: 0x6aad9cf3, blocks: 0x22809 00010000:00010000:1.0:1789762803.888507:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.888520:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762803.888523:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.888528:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.888541:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762803.901633:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967879 is committed 00002000:00010000:1.0:1789762803.901648:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 72351744 -> 73400320 00002000:00010000:1.0:1789762803.901655:0:6653:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb mtime from disk: 1789762803 -> 1789762804 00002000:00010000:1.0:1789762803.901658:0:6653:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb ctime from disk: 1789762803 -> 1789762804 00002000:00010000:1.0:1789762803.901661:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 141321 -> 143369 00010000:00010000:1.0:1789762803.901675:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.901690:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.901700:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.901708:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.901715:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.901725:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.901746:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.901754:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.901873:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014992256:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13427us (45682us total) trans 4294967879 rc 0/0 00000100:00100000:1.0:1789762803.901885:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 562 00000100:00100000:1.0:1789762803.901898:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327192215981 time 327192316585 state 0x2 ref 0 00000100:00100000:3.0:1789762803.904192:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762803.904225:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014992384 00000100:00100000:1.0:1789762803.904234:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014992384 00000100:00100000:1.0:1789762803.904275:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327192215981 time 327192316585 state 0x0 ref 1 00000100:00100000:1.0:1789762803.904286:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327242316585 time 327192316585 state 0x1 ref 1 00000100:00100000:1.0:1789762803.938518:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327242316585 time 327242412936 state 0x0 ref 1 00000100:00100000:1.0:1789762803.938544:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014992384:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.938597:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 73400320, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x23009 00010000:00010000:1.0:1789762803.938604:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e59 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.938618:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762803.938622:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.938626:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.938637:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762803.951550:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 73400320 -> 74448896 00000001:00080000:2.0:1789762803.951550:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967880 is committed 00002000:00010000:1.0:1789762803.951557:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 143369 -> 145417 00010000:00010000:1.0:1789762803.951569:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.951580:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.951588:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.951593:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.951596:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.951609:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.951629:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.951638:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.951715:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc02680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014992384:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13179us (47521us total) trans 4294967880 rc 0/0 00000100:00100000:1.0:1789762803.951724:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 563 00000100:00100000:1.0:1789762803.951733:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327242316585 time 327242412936 state 0x2 ref 0 00000100:00100000:2.0:1789762803.953897:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762803.953953:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014992512 00000100:00100000:0.0:1789762803.953962:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014992512 00000100:00100000:0.0:1789762803.954008:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327242316585 time 327242412936 state 0x0 ref 1 00000100:00100000:0.0:1789762803.954022:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327292412936 time 327242412936 state 0x1 ref 1 00000100:00100000:1.0:1789762803.988619:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327292412936 time 327292512396 state 0x0 ref 1 00000100:00100000:1.0:1789762803.988649:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014992512:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762803.988700:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 74448896, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x23809 00010000:00010000:1.0:1789762803.988708:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.988725:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762803.988731:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762803.988737:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.988753:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762803.999628:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967881 is committed 00002000:00010000:1.0:1789762803.999656:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:1.0:1789762803.999661:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 145417 -> 147465 00010000:00010000:1.0:1789762803.999670:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.999679:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.999686:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.999691:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762803.999695:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.999701:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.999715:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762803.999720:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762803.999783:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014992512:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11144us (45884us total) trans 4294967881 rc 0/0 00000100:00100000:1.0:1789762803.999794:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 564 00000100:00100000:1.0:1789762803.999807:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327292412936 time 327292512396 state 0x2 ref 0 00000100:00100000:0.0:1789762804.002140:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762804.002174:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014992640 00000100:00100000:1.0:1789762804.002183:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014992640 00000100:00100000:1.0:1789762804.002220:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327292412936 time 327292512396 state 0x0 ref 1 00000100:00100000:1.0:1789762804.002231:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327342512396 time 327292512396 state 0x1 ref 1 00000100:00100000:1.0:1789762804.038723:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327342512396 time 327342619579 state 0x0 ref 1 00000100:00100000:1.0:1789762804.038748:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014992640:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.038796:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 75497472, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x24009 00010000:00010000:1.0:1789762804.038803:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.038816:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.038820:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.038825:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.038835:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762804.050675:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967882 is committed 00002000:00010000:1.0:1789762804.050742:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:1.0:1789762804.050747:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 147465 -> 149513 00010000:00010000:1.0:1789762804.050758:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.050767:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.050775:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.050783:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.050788:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.050798:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.050818:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.050827:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.050896:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014992640:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12156us (48755us total) trans 4294967882 rc 0/0 00000100:00100000:1.0:1789762804.050906:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 565 00000100:00100000:1.0:1789762804.050919:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327342512396 time 327342619579 state 0x2 ref 0 00000100:00100000:3.0:1789762804.052977:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762804.053010:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014992768 00000100:00100000:1.0:1789762804.053018:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014992768 00000100:00100000:1.0:1789762804.053055:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327342512396 time 327342619579 state 0x0 ref 1 00000100:00100000:1.0:1789762804.053065:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327392619579 time 327342619579 state 0x1 ref 1 00000100:00100000:1.0:1789762804.088785:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327392619579 time 327392681441 state 0x0 ref 1 00000100:00100000:1.0:1789762804.088811:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014992768:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.088865:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 76546048, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x24809 00010000:00010000:1.0:1789762804.088873:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.088887:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.088891:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.088895:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.088906:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762804.100391:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967883 is committed 00002000:00010000:1.0:1789762804.100426:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:1.0:1789762804.100431:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 149513 -> 151561 00010000:00010000:1.0:1789762804.100440:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.100449:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.100456:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.100461:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.100465:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.100472:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.100486:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.100491:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.100548:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc00a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014992768:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11746us (47571us total) trans 4294967883 rc 0/0 00000100:00100000:1.0:1789762804.100557:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 566 00000100:00100000:1.0:1789762804.100565:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327392619579 time 327392681441 state 0x2 ref 0 00000100:00100000:3.0:1789762804.102936:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762804.103011:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014992896 00000100:00100000:0.0:1789762804.103019:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014992896 00000100:00100000:0.0:1789762804.103058:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327392619579 time 327392681441 state 0x0 ref 1 00000100:00100000:0.0:1789762804.103069:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327442681441 time 327392681441 state 0x1 ref 1 00000100:00100000:1.0:1789762804.138888:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327442681441 time 327442777489 state 0x0 ref 1 00000100:00100000:1.0:1789762804.138919:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc00700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014992896:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.138975:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 77594624, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x25009 00010000:00010000:1.0:1789762804.138986:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.139003:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.139008:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.139015:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.139029:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762804.150624:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967884 is committed 00002000:00010000:1.0:1789762804.150654:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:1.0:1789762804.150661:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 151561 -> 153609 00010000:00010000:1.0:1789762804.150674:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.150687:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.150698:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.150707:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.150712:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.150722:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.150741:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.150750:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.150828:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc00700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014992896:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11917us (47888us total) trans 4294967884 rc 0/0 00000100:00100000:1.0:1789762804.150839:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 567 00000100:00100000:1.0:1789762804.150851:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327442681441 time 327442777489 state 0x2 ref 0 00000100:00100000:0.0:1789762804.153200:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762804.153240:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014993024 00000100:00100000:1.0:1789762804.153249:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014993024 00000100:00100000:1.0:1789762804.153293:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327442681441 time 327442777489 state 0x0 ref 1 00000100:00100000:1.0:1789762804.153312:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327492777489 time 327442777489 state 0x1 ref 1 00000100:00100000:1.0:1789762804.188976:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327492777489 time 327492869122 state 0x0 ref 1 00000100:00100000:1.0:1789762804.189009:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014993024:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.189067:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 78643200, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x25809 00010000:00010000:1.0:1789762804.189075:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.189116:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.189120:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.189125:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.189137:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762804.200473:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967885 is committed 00002000:00010000:1.0:1789762804.200489:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:1.0:1789762804.200495:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 153609 -> 155657 00010000:00010000:1.0:1789762804.200505:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.200515:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.200527:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.200532:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.200536:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.200542:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.200556:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.200561:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.200627:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014993024:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11630us (47425us total) trans 4294967885 rc 0/0 00000100:00100000:1.0:1789762804.200634:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 568 00000100:00100000:1.0:1789762804.200644:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327492777489 time 327492869122 state 0x2 ref 0 00000100:00100000:3.0:1789762804.203064:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762804.203116:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014993152 00000100:00100000:2.0:1789762804.203125:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014993152 00000100:00100000:2.0:1789762804.203164:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327492777489 time 327492869122 state 0x0 ref 1 00000100:00100000:2.0:1789762804.203185:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327542869122 time 327492869122 state 0x1 ref 1 00000100:00100000:1.0:1789762804.239075:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327542869122 time 327542971442 state 0x0 ref 1 00000100:00100000:1.0:1789762804.239125:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014993152:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.239175:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 79691776, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x26009 00010000:00010000:1.0:1789762804.239183:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.239197:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.239200:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.239205:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.239216:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762804.251226:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967886 is committed 00002000:00010000:1.0:1789762804.251233:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:1.0:1789762804.251238:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 155657 -> 157705 00010000:00010000:1.0:1789762804.251250:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.251260:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.251267:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.251273:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.251276:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.251291:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.251310:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.251320:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.251399:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014993152:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12284us (48334us total) trans 4294967886 rc 0/0 00000100:00100000:1.0:1789762804.251407:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 569 00000100:00100000:1.0:1789762804.251417:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327542869122 time 327542971442 state 0x2 ref 0 00000100:00100000:3.0:1789762804.253729:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762804.253789:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014993280 00000100:00100000:0.0:1789762804.253798:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014993280 00000100:00100000:0.0:1789762804.253837:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327542869122 time 327542971442 state 0x0 ref 1 00000100:00100000:0.0:1789762804.253857:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327592971442 time 327542971442 state 0x1 ref 1 00000100:00100000:1.0:1789762804.289192:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327592971442 time 327593089057 state 0x0 ref 1 00000100:00100000:1.0:1789762804.289214:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014993280:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.289261:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 80740352, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x26809 00010000:00010000:1.0:1789762804.289268:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.289281:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.289285:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.289289:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.289301:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762804.300247:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967887 is committed 00002000:00010000:1.0:1789762804.300263:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 80740352 -> 81788928 00002000:00010000:1.0:1789762804.300268:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 157705 -> 159753 00010000:00010000:1.0:1789762804.300278:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.300287:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.300294:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.300299:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.300302:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.300309:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.300322:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.300341:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.300400:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014993280:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11192us (46669us total) trans 4294967887 rc 0/0 00000100:00100000:1.0:1789762804.300408:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 570 00000100:00100000:1.0:1789762804.300417:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327592971442 time 327593089057 state 0x2 ref 0 00000100:00100000:3.0:1789762804.302597:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762804.302629:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014993408 00000100:00100000:1.0:1789762804.302636:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014993408 00000100:00100000:1.0:1789762804.302674:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327592971442 time 327593089057 state 0x0 ref 1 00000100:00100000:1.0:1789762804.302690:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327643089057 time 327593089057 state 0x1 ref 1 00000100:00100000:1.0:1789762804.339264:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327643089057 time 327643160967 state 0x0 ref 1 00000100:00100000:1.0:1789762804.339290:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014993408:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.339352:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 81788928, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x27009 00010000:00010000:1.0:1789762804.339360:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.339374:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.339377:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.339382:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.339393:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762804.351035:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967888 is committed 00002000:00010000:1.0:1789762804.351062:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:1.0:1789762804.351067:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 159753 -> 161801 00010000:00010000:1.0:1789762804.351078:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.351102:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.351110:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.351115:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.351119:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.351125:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.351140:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.351146:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.351208:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014993408:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11926us (48609us total) trans 4294967888 rc 0/0 00000100:00100000:1.0:1789762804.351218:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 571 00000100:00100000:1.0:1789762804.351227:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327643089057 time 327643160967 state 0x2 ref 0 00000100:00100000:3.0:1789762804.353504:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762804.353556:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014993536 00000100:00100000:2.0:1789762804.353564:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014993536 00000100:00100000:2.0:1789762804.353613:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327643089057 time 327643160967 state 0x0 ref 1 00000100:00100000:2.0:1789762804.353629:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327693160967 time 327643160967 state 0x1 ref 1 00000100:00100000:1.0:1789762804.389372:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327693160967 time 327693262482 state 0x0 ref 1 00000100:00100000:1.0:1789762804.389397:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014993536:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.389445:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 82837504, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x27809 00010000:00010000:1.0:1789762804.389452:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.389466:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.389470:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.389474:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.389485:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762804.400910:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967889 is committed 00002000:00010000:1.0:1789762804.400966:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 82837504 -> 83886080 00002000:00010000:1.0:1789762804.400971:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 161801 -> 163849 00010000:00010000:1.0:1789762804.400982:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.400995:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.401006:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.401015:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.401020:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.401031:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.401046:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.401052:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.401139:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014993536:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11748us (47634us total) trans 4294967889 rc 0/0 00000100:00100000:1.0:1789762804.401148:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 572 00000100:00100000:1.0:1789762804.401156:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327693160967 time 327693262482 state 0x2 ref 0 00000100:00100000:2.0:1789762804.403361:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762804.403406:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014993664 00000100:00100000:0.0:1789762804.403415:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014993664 00000100:00100000:0.0:1789762804.403452:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327693160967 time 327693262482 state 0x0 ref 1 00000100:00100000:0.0:1789762804.403464:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327743262482 time 327693262482 state 0x1 ref 1 00000100:00100000:1.0:1789762804.439494:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327743262482 time 327743386481 state 0x0 ref 1 00000100:00100000:1.0:1789762804.439522:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfedd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014993664:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.439580:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 83886080, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x28009 00010000:00010000:1.0:1789762804.439591:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.439609:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.439614:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.439621:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.439637:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762804.450964:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967890 is committed 00002000:00010000:1.0:1789762804.451034:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 83886080 -> 84934656 00002000:00010000:1.0:1789762804.451041:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 163849 -> 165897 00010000:00010000:1.0:1789762804.451053:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.451067:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.451079:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.451110:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.451116:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.451134:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.451153:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.451162:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03e9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.451238:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfedd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014993664:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11724us (47875us total) trans 4294967890 rc 0/0 00000100:00100000:1.0:1789762804.451251:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 573 00000100:00100000:1.0:1789762804.451264:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327743262482 time 327743386481 state 0x2 ref 0 00000100:00100000:3.0:1789762804.453401:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762804.453435:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014993792 00000100:00100000:1.0:1789762804.453444:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014993792 00000100:00100000:1.0:1789762804.453481:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327743262482 time 327743386481 state 0x0 ref 1 00000100:00100000:1.0:1789762804.453493:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327793386481 time 327743386481 state 0x1 ref 1 00000100:00100000:1.0:1789762804.489569:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327793386481 time 327793463625 state 0x0 ref 1 00000100:00100000:1.0:1789762804.489602:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014993792:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.489664:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 84934656, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x28809 00010000:00010000:1.0:1789762804.489674:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ea6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.489690:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.489695:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.489702:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ea6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.489717:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ea6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762804.501145:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967891 is committed 00002000:00010000:1.0:1789762804.501147:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:1.0:1789762804.501153:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 165897 -> 167945 00010000:00010000:1.0:1789762804.501164:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ea6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.501174:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ea6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.501181:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ea6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.501187:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.501191:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ea6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.501202:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ea6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.501217:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ea6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.501222:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ea6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.501285:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014993792:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11696us (47883us total) trans 4294967891 rc 0/0 00000100:00100000:1.0:1789762804.501294:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 574 00000100:00100000:1.0:1789762804.501305:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327793386481 time 327793463625 state 0x2 ref 0 00000100:00100000:3.0:1789762804.503615:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762804.503666:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014993920 00000100:00100000:2.0:1789762804.503674:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014993920 00000100:00100000:2.0:1789762804.503714:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327793386481 time 327793463625 state 0x0 ref 1 00000100:00100000:2.0:1789762804.503725:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327843463625 time 327793463625 state 0x1 ref 1 00000100:00100000:1.0:1789762804.539662:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327843463625 time 327843551094 state 0x0 ref 1 00000100:00100000:1.0:1789762804.539689:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014993920:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.539739:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 85983232, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x29009 00010000:00010000:1.0:1789762804.539749:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ead lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.539766:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.539771:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.539778:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ead lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.539795:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ead lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762804.551452:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967892 is committed 00002000:00010000:1.0:1789762804.551462:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 85983232 -> 87031808 00002000:00010000:1.0:1789762804.551468:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 167945 -> 169993 00010000:00010000:1.0:1789762804.551476:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ead lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.551486:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ead lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.551493:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ead lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.551499:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.551502:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ead lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.551508:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ead lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.551522:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ead lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.551527:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03ead lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.551582:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014993920:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11904us (47967us total) trans 4294967892 rc 0/0 00000100:00100000:1.0:1789762804.551591:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 575 00000100:00100000:1.0:1789762804.551599:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327843463625 time 327843551094 state 0x2 ref 0 00000100:00100000:3.0:1789762804.553843:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762804.553897:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014994048 00000100:00100000:0.0:1789762804.553907:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014994048 00000100:00100000:0.0:1789762804.553949:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327843463625 time 327843551094 state 0x0 ref 1 00000100:00100000:0.0:1789762804.553961:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327893551094 time 327843551094 state 0x1 ref 1 00000100:00100000:1.0:1789762804.589772:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327893551094 time 327893669512 state 0x0 ref 1 00000100:00100000:1.0:1789762804.589793:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014994048:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.589833:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 87031808, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x29809 00010000:00010000:1.0:1789762804.589840:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03eb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.589857:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.589862:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.589870:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03eb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.589885:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03eb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762804.601238:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967893 is committed 00002000:00010000:3.0:1789762804.601301:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 87031808 -> 88080384 00002000:00010000:3.0:1789762804.601308:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 169993 -> 172041 00010000:00010000:3.0:1789762804.601321:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03eb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.601346:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03eb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.601370:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03eb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.601379:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762804.601384:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03eb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.601395:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03eb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.601415:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03eb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.601422:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b03eb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762804.601494:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc02300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014994048:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11706us (47649us total) trans 4294967893 rc 0/0 00000100:00100000:3.0:1789762804.601502:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 576 00000100:00100000:3.0:1789762804.601510:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327893551094 time 327893669512 state 0x2 ref 0 00000100:00100000:3.0:1789762804.603587:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762804.603644:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014994176 00000100:00100000:1.0:1789762804.603652:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014994176 00000100:00100000:1.0:1789762804.603688:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327893551094 time 327893669512 state 0x0 ref 1 00000100:00100000:1.0:1789762804.603700:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327943669512 time 327893669512 state 0x1 ref 1 00000100:00100000:1.0:1789762804.639846:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327943669512 time 327943741684 state 0x0 ref 1 00000100:00100000:1.0:1789762804.639871:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014994176:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.639923:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 88080384, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x2a009 00010000:00010000:1.0:1789762804.639930:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ebb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.639943:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.639946:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.639951:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ebb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.639962:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ebb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762804.652522:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967894 is committed 00002000:00010000:1.0:1789762804.652585:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 88080384 -> 89128960 00002000:00010000:1.0:1789762804.652591:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 172041 -> 174089 00010000:00010000:1.0:1789762804.652604:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ebb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.652618:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ebb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.652634:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ebb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.652640:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762804.652644:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ebb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.652651:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ebb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.652665:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ebb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.652671:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ebb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762804.652738:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014994176:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12875us (49149us total) trans 4294967894 rc 0/0 00000100:00100000:1.0:1789762804.652746:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 577 00000100:00100000:1.0:1789762804.652756:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327943669512 time 327943741684 state 0x2 ref 0 00000100:00100000:3.0:1789762804.655234:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762804.655310:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014994304 00000100:00100000:2.0:1789762804.655319:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014994304 00000100:00100000:2.0:1789762804.655372:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327943669512 time 327943741684 state 0x0 ref 1 00000100:00100000:2.0:1789762804.655384:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327993741684 time 327943741684 state 0x1 ref 1 00010000:00010000:1.0:1789762804.663212:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762804.689949:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327993741684 time 327993845469 state 0x0 ref 1 00000100:00100000:1.0:1789762804.689974:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014994304:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.690025:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 89128960, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x2a809 00010000:00010000:1.0:1789762804.690032:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ec2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.690046:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=327s v=(0 0) 00010000:00010000:1.0:1789762804.690050:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.690054:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ec2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.690065:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ec2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762804.701337:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967895 is committed 00002000:00010000:3.0:1789762804.701382:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 89128960 -> 90177536 00002000:00010000:3.0:1789762804.701386:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 174089 -> 176137 00010000:00010000:3.0:1789762804.701396:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ec2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.701406:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ec2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.701412:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ec2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.701417:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762804.701421:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ec2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.701427:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ec2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.701442:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ec2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.701447:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b03ec2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762804.701505:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014994304:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11540us (46271us total) trans 4294967895 rc 0/0 00000100:00100000:3.0:1789762804.701513:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 578 00000100:00100000:3.0:1789762804.701528:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327993741684 time 327993845469 state 0x2 ref 0 00000100:00100000:3.0:1789762804.703914:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762804.703993:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014994432 00000100:00100000:0.0:1789762804.704001:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014994432 00000100:00100000:0.0:1789762804.704038:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 327993741684 time 327993845469 state 0x0 ref 1 00000100:00100000:0.0:1789762804.704048:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328043845469 time 327993845469 state 0x1 ref 1 00000100:00100000:3.0:1789762804.740108:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328043845469 time 328043982249 state 0x0 ref 1 00000100:00100000:3.0:1789762804.740132:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014994432:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762804.740180:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 90177536, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x2b009 00010000:00010000:3.0:1789762804.740187:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ec9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.740200:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:3.0:1789762804.740204:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762804.740209:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ec9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.740219:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ec9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762804.751656:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967896 is committed 00002000:00010000:3.0:1789762804.751714:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 90177536 -> 91226112 00002000:00010000:3.0:1789762804.751719:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 176137 -> 178185 00010000:00010000:3.0:1789762804.751729:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ec9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.751738:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ec9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.751756:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ec9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.751762:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762804.751765:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ec9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.751771:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ec9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.751787:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ec9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.751793:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ec9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762804.751853:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014994432:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11727us (47938us total) trans 4294967896 rc 0/0 00000100:00100000:3.0:1789762804.751862:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 579 00000100:00100000:3.0:1789762804.751870:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328043845469 time 328043982249 state 0x2 ref 0 00000100:00100000:3.0:1789762804.754143:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762804.754187:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014994560 00000100:00100000:1.0:1789762804.754196:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014994560 00000100:00100000:1.0:1789762804.754234:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328043845469 time 328043982249 state 0x0 ref 1 00000100:00100000:1.0:1789762804.754245:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328093982249 time 328043982249 state 0x1 ref 1 00000100:00100000:1.0:1789762804.790128:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328093982249 time 328094030674 state 0x0 ref 1 00000100:00100000:1.0:1789762804.790143:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014994560:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762804.790175:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 91226112, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x2b809 00010000:00010000:1.0:1789762804.790182:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b03ed0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.790197:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:1.0:1789762804.790201:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762804.790205:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b03ed0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762804.790214:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b03ed0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762804.801529:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967897 is committed 00002000:00010000:3.0:1789762804.801555:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 91226112 -> 92274688 00002000:00010000:3.0:1789762804.801560:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 178185 -> 180233 00010000:00010000:3.0:1789762804.801569:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b03ed0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.801587:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b03ed0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.801593:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b03ed0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.801599:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762804.801602:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b03ed0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.801608:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b03ed0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.801621:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b03ed0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.801626:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b03ed0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762804.801685:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014994560:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11543us (47540us total) trans 4294967897 rc 0/0 00000100:00100000:3.0:1789762804.801693:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 580 00000100:00100000:3.0:1789762804.801701:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328093982249 time 328094030674 state 0x2 ref 0 00000100:00100000:3.0:1789762804.803713:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762804.803764:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014994688 00000100:00100000:2.0:1789762804.803770:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014994688 00000100:00100000:2.0:1789762804.803806:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328093982249 time 328094030674 state 0x0 ref 1 00000100:00100000:2.0:1789762804.803825:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328144030674 time 328094030674 state 0x1 ref 1 00000100:00100000:3.0:1789762804.840251:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328144030674 time 328144147498 state 0x0 ref 1 00000100:00100000:3.0:1789762804.840275:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014994688:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762804.840344:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 92274688, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x2c009 00010000:00010000:3.0:1789762804.840353:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ed7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.840366:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:3.0:1789762804.840369:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762804.840375:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ed7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.840386:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ed7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762804.852766:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967898 is committed 00002000:00010000:3.0:1789762804.852791:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:3.0:1789762804.852797:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 180233 -> 182281 00010000:00010000:3.0:1789762804.852808:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ed7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.852818:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ed7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.852825:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ed7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.852841:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762804.852845:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ed7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.852852:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ed7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.852867:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ed7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.852873:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ed7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762804.852944:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014994688:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12674us (49228us total) trans 4294967898 rc 0/0 00000100:00100000:3.0:1789762804.852952:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 581 00000100:00100000:3.0:1789762804.852961:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328144030674 time 328144147498 state 0x2 ref 0 00000100:00100000:2.0:1789762804.855362:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762804.855401:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014994816 00000100:00100000:0.0:1789762804.855410:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014994816 00000100:00100000:0.0:1789762804.855458:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328144030674 time 328144147498 state 0x0 ref 1 00000100:00100000:0.0:1789762804.855474:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328194147498 time 328144147498 state 0x1 ref 1 00000001:00080000:0.0:1789762804.870374:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967397 is committed 00000001:00080000:0.0:1789762804.870388:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967390 is committed 00000001:00080000:0.0:1789762804.870405:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967395 is committed 00000001:00080000:0.0:1789762804.870437:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967396 is committed 00000100:00100000:3.0:1789762804.890346:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328194147498 time 328194233217 state 0x0 ref 1 00000100:00100000:3.0:1789762804.890370:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfede300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014994816:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762804.890423:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 93323264, mtime: 0x6aad9cf4, atime: 0x0, ctime: 0x6aad9cf4, blocks: 0x2c809 00010000:00010000:3.0:1789762804.890432:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ede lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.890454:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:3.0:1789762804.890457:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762804.890462:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ede lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.890474:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ede lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762804.901224:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967899 is committed 00002000:00010000:3.0:1789762804.901242:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:3.0:1789762804.901247:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb mtime from disk: 1789762804 -> 1789762805 00002000:00010000:3.0:1789762804.901249:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb ctime from disk: 1789762804 -> 1789762805 00002000:00010000:3.0:1789762804.901251:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 182281 -> 184329 00010000:00010000:3.0:1789762804.901261:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ede lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.901274:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ede lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.901284:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ede lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.901293:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762804.901299:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ede lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.901318:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ede lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.901348:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ede lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.901354:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ede lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762804.901420:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfede300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014994816:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11056us (46057us total) trans 4294967899 rc 0/0 00000100:00100000:3.0:1789762804.901428:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 582 00000100:00100000:3.0:1789762804.901436:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328194147498 time 328194233217 state 0x2 ref 0 00000100:00100000:2.0:1789762804.903716:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762804.903791:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014994944 00000100:00100000:1.0:1789762804.903799:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014994944 00000100:00100000:1.0:1789762804.903837:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328194147498 time 328194233217 state 0x0 ref 1 00000100:00100000:1.0:1789762804.903848:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328244233217 time 328194233217 state 0x1 ref 1 00000100:00100000:3.0:1789762804.940440:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328244233217 time 328244336773 state 0x0 ref 1 00000100:00100000:3.0:1789762804.940463:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014994944:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762804.940509:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 94371840, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x2d009 00010000:00010000:3.0:1789762804.940516:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ee5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.940529:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:3.0:1789762804.940532:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762804.940537:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ee5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.940561:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ee5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762804.952146:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 94371840 -> 95420416 00002000:00010000:3.0:1789762804.952152:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 184329 -> 186377 00010000:00010000:3.0:1789762804.952163:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ee5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762804.952172:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967900 is committed 00010000:00010000:3.0:1789762804.952173:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ee5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.952184:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ee5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.952190:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762804.952194:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ee5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.952201:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ee5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.952215:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ee5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.952220:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ee5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762804.952288:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702014994944:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11832us (48570us total) trans 4294967900 rc 0/0 00000100:00100000:3.0:1789762804.952297:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 583 00000100:00100000:3.0:1789762804.952307:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328244233217 time 328244336773 state 0x2 ref 0 00000100:00100000:2.0:1789762804.954378:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762804.954414:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014995072 00000100:00100000:3.0:1789762804.954422:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014995072 00000100:00100000:3.0:1789762804.954466:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328244233217 time 328244336773 state 0x0 ref 1 00000100:00100000:3.0:1789762804.954498:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328294336773 time 328244336773 state 0x1 ref 1 00000100:00100000:3.0:1789762804.990524:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328294336773 time 328294420394 state 0x0 ref 1 00000100:00100000:3.0:1789762804.990549:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014995072:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762804.990599:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 95420416, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x2d809 00010000:00010000:3.0:1789762804.990606:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03eec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.990619:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:3.0:1789762804.990623:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762804.990628:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03eec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762804.990638:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03eec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762805.002597:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967901 is committed 00002000:00010000:3.0:1789762805.002650:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 95420416 -> 96468992 00002000:00010000:3.0:1789762805.002655:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 186377 -> 188425 00010000:00010000:3.0:1789762805.002667:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03eec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.002677:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03eec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.002684:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03eec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.002689:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762805.002692:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03eec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.002699:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03eec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.002713:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03eec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.002718:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03eec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762805.002788:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702014995072:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12246us (48410us total) trans 4294967901 rc 0/0 00000100:00100000:3.0:1789762805.002804:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 584 00000100:00100000:3.0:1789762805.002814:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328294336773 time 328294420394 state 0x2 ref 0 00000100:00100000:3.0:1789762805.005162:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.005216:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014995200 00000100:00100000:1.0:1789762805.005225:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014995200 00000100:00100000:1.0:1789762805.005284:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328294336773 time 328294420394 state 0x0 ref 1 00000100:00080000:2.0:1789762805.005287:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762805.005301:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328344420394 time 328294420394 state 0x1 ref 1 00000100:00100000:3.0:1789762805.040635:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328344420394 time 328344526356 state 0x0 ref 1 00000100:00100000:3.0:1789762805.040662:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014995200:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762805.040709:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 96468992, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x2e009 00010000:00010000:3.0:1789762805.040716:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ef3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.040730:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:3.0:1789762805.040734:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762805.040739:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ef3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.040750:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ef3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762805.052884:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967902 is committed 00002000:00010000:1.0:1789762805.052918:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 96468992 -> 97517568 00002000:00010000:1.0:1789762805.052925:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 188425 -> 190473 00010000:00010000:1.0:1789762805.052937:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ef3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.052956:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ef3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.052967:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ef3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.052975:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.052980:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ef3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.052990:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ef3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.053010:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ef3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.053020:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b03ef3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.053139:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014995200:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12481us (47971us total) trans 4294967902 rc 0/0 00000100:00100000:1.0:1789762805.053151:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 585 00000100:00100000:1.0:1789762805.053164:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328344420394 time 328344526356 state 0x2 ref 0 00000100:00100000:3.0:1789762805.055812:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.055860:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014995328 00000100:00100000:1.0:1789762805.055869:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014995328 00000100:00100000:1.0:1789762805.055912:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328344420394 time 328344526356 state 0x0 ref 1 00000100:00100000:1.0:1789762805.055928:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328394526356 time 328344526356 state 0x1 ref 1 00000100:00100000:3.0:1789762805.090691:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328394526356 time 328394588105 state 0x0 ref 1 00000100:00100000:3.0:1789762805.090713:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014995328:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762805.090783:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 97517568, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x2e809 00010000:00010000:3.0:1789762805.090791:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03efa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.090812:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:3.0:1789762805.090816:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762805.090821:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03efa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.090833:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03efa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762805.103706:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967903 is committed 00002000:00010000:3.0:1789762805.103726:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 97517568 -> 98566144 00002000:00010000:3.0:1789762805.103733:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 190473 -> 192521 00010000:00010000:3.0:1789762805.103746:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03efa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.103761:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03efa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.103769:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03efa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.103775:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762805.103778:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03efa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.103796:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03efa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.103811:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03efa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.103816:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03efa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762805.103877:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014995328:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13169us (48067us total) trans 4294967903 rc 0/0 00000100:00100000:3.0:1789762805.103886:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 586 00000100:00100000:3.0:1789762805.103895:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328394526356 time 328394588105 state 0x2 ref 0 00000100:00100000:3.0:1789762805.106503:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762805.106546:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014995456 00000100:00100000:2.0:1789762805.106554:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014995456 00000100:00100000:2.0:1789762805.106592:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328394526356 time 328394588105 state 0x0 ref 1 00000100:00100000:2.0:1789762805.106603:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328444588105 time 328394588105 state 0x1 ref 1 00000100:00100000:3.0:1789762805.140778:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328444588105 time 328444671148 state 0x0 ref 1 00000100:00100000:3.0:1789762805.140809:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014995456:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762805.140870:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 98566144, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x2f009 00010000:00010000:3.0:1789762805.140877:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03f01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.140891:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:3.0:1789762805.140895:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762805.140915:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03f01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.140927:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03f01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762805.153561:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967904 is committed 00002000:00010000:1.0:1789762805.153593:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 98566144 -> 99614720 00002000:00010000:1.0:1789762805.153599:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 192521 -> 194569 00010000:00010000:1.0:1789762805.153612:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03f01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.153623:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03f01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.153633:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03f01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.153640:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.153645:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03f01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.153654:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03f01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.153674:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03f01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.153682:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b03f01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.153759:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014995456:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12958us (47250us total) trans 4294967904 rc 0/0 00000100:00100000:1.0:1789762805.153770:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 587 00000100:00100000:1.0:1789762805.153782:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328444588105 time 328444671148 state 0x2 ref 0 00000100:00100000:2.0:1789762805.156481:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.156519:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014995584 00000100:00100000:1.0:1789762805.156528:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014995584 00000100:00100000:1.0:1789762805.156572:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328444588105 time 328444671148 state 0x0 ref 1 00000100:00100000:1.0:1789762805.156587:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328494671148 time 328444671148 state 0x1 ref 1 00000100:00100000:1.0:1789762805.190854:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328494671148 time 328494747925 state 0x0 ref 1 00000100:00100000:1.0:1789762805.190882:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc3c1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014995584:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762805.190941:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 99614720, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x2f809 00010000:00010000:1.0:1789762805.190950:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.190964:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:1.0:1789762805.190969:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.190975:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.190988:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762805.204543:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967905 is committed 00002000:00010000:1.0:1789762805.204577:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 99614720 -> 100663296 00002000:00010000:1.0:1789762805.204583:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 194569 -> 196619 00010000:00010000:1.0:1789762805.204596:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.204608:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.204617:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.204625:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.204629:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.204639:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.204657:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.204665:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.204753:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc3c1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014995584:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13875us (48266us total) trans 4294967905 rc 0/0 00000100:00100000:1.0:1789762805.204766:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 588 00000100:00100000:1.0:1789762805.204778:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328494671148 time 328494747925 state 0x2 ref 0 00000100:00100000:3.0:1789762805.207416:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.207460:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014995712 00000100:00100000:1.0:1789762805.207469:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014995712 00000100:00100000:1.0:1789762805.207516:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328494671148 time 328494747925 state 0x0 ref 1 00000100:00100000:1.0:1789762805.207531:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328544747925 time 328494747925 state 0x1 ref 1 00000100:00100000:3.0:1789762805.240960:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328544747925 time 328544855535 state 0x0 ref 1 00000100:00100000:3.0:1789762805.240984:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014995712:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762805.241034:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 100663296, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x3000b 00010000:00010000:3.0:1789762805.241041:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.241055:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:3.0:1789762805.241059:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762805.241063:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.241074:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762805.253121:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967906 is committed 00002000:00010000:3.0:1789762805.253129:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 100663296 -> 101711872 00002000:00010000:3.0:1789762805.253135:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 196619 -> 198667 00010000:00010000:3.0:1789762805.253146:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.253168:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.253175:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.253179:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762805.253183:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.253190:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.253204:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.253209:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762805.253274:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014995712:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12297us (45858us total) trans 4294967906 rc 0/0 00000100:00100000:3.0:1789762805.253283:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 589 00000100:00100000:3.0:1789762805.253294:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328544747925 time 328544855535 state 0x2 ref 0 00000100:00100000:3.0:1789762805.255429:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762805.255490:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014995840 00000100:00100000:2.0:1789762805.255499:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014995840 00000100:00100000:2.0:1789762805.255537:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328544747925 time 328544855535 state 0x0 ref 1 00000100:00100000:2.0:1789762805.255548:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328594855535 time 328544855535 state 0x1 ref 1 00000100:00100000:1.0:1789762805.291052:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328594855535 time 328594943274 state 0x0 ref 1 00000100:00100000:1.0:1789762805.291080:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014995840:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762805.291159:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 101711872, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x3080b 00010000:00010000:1.0:1789762805.291169:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.291185:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:1.0:1789762805.291190:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.291196:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.291210:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762805.303201:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967907 is committed 00002000:00010000:2.0:1789762805.303214:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:2.0:1789762805.303220:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 198667 -> 200715 00010000:00010000:2.0:1789762805.303233:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.303244:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.303251:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.303256:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762805.303259:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.303266:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.303281:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.303286:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b03f16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762805.303377:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014995840:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12305us (47945us total) trans 4294967907 rc 0/0 00000100:00100000:2.0:1789762805.303394:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 590 00000100:00100000:2.0:1789762805.303405:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328594855535 time 328594943274 state 0x2 ref 0 00000100:00100000:3.0:1789762805.305458:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.305512:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014995968 00000100:00100000:1.0:1789762805.305521:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014995968 00000100:00100000:1.0:1789762805.305566:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328594855535 time 328594943274 state 0x0 ref 1 00000100:00100000:1.0:1789762805.305581:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328644943274 time 328594943274 state 0x1 ref 1 00000100:00100000:1.0:1789762805.341148:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328644943274 time 328645039000 state 0x0 ref 1 00000100:00100000:1.0:1789762805.341178:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014995968:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762805.341233:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 102760448, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x3100b 00010000:00010000:1.0:1789762805.341243:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.341259:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:1.0:1789762805.341264:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.341271:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.341285:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762805.353236:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967908 is committed 00002000:00010000:1.0:1789762805.353260:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:1.0:1789762805.353266:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 200715 -> 202763 00010000:00010000:1.0:1789762805.353280:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.353294:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.353303:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.353312:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.353318:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.353340:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.353362:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.353371:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.353455:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014995968:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12284us (47995us total) trans 4294967908 rc 0/0 00000100:00100000:1.0:1789762805.353467:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 591 00000100:00100000:1.0:1789762805.353480:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328644943274 time 328645039000 state 0x2 ref 0 00000100:00100000:3.0:1789762805.355823:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.355866:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014996096 00000100:00100000:1.0:1789762805.355876:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014996096 00000100:00100000:1.0:1789762805.355921:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328644943274 time 328645039000 state 0x0 ref 1 00000100:00100000:1.0:1789762805.355936:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328695039000 time 328645039000 state 0x1 ref 1 00000100:00100000:3.0:1789762805.391229:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328695039000 time 328695124039 state 0x0 ref 1 00000100:00100000:3.0:1789762805.391250:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014996096:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762805.391299:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 103809024, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x3180b 00010000:00010000:3.0:1789762805.391306:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.391332:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:3.0:1789762805.391337:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762805.391344:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.391356:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762805.403491:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967909 is committed 00002000:00010000:1.0:1789762805.403520:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb size from disk: 103809024 -> 104857600 00002000:00010000:1.0:1789762805.403527:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x11:0x0] updating lvb blocks from disk: 202763 -> 204811 00010000:00010000:1.0:1789762805.403540:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.403554:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.403563:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.403570:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.403575:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.403585:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.403605:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.403613:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b03f24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.403697:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702014996096:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12450us (47871us total) trans 4294967909 rc 0/0 00000100:00100000:1.0:1789762805.403708:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 592 00000100:00100000:1.0:1789762805.403721:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328695039000 time 328695124039 state 0x2 ref 0 00000100:00100000:0.0:1789762805.405039:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762805.405082:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014996352 00000100:00100000:3.0:1789762805.405105:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014996352 00000100:00080000:2.0:1789762805.405161:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762805.405161:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 79 00000100:00100000:3.0:1789762805.405172:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:13702:x1876702014996352:12345-192.168.203.48@tcp:35:dd.500 00000001:00080000:3.0:1789762805.405394:0:5807:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967513 00000100:00100000:3.0:1789762805.405460:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13702:x1876702014996352:12345-192.168.203.48@tcp:35:dd.500 Request processed in 290us (422us total) trans 4294967514 rc 0/0 00000100:00100000:3.0:1789762805.405469:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 79 00000020:00080000:3.0:1789762805.405473:0:5807:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1789762805.453569:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.453634:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014996608 00000100:00100000:1.0:1789762805.453645:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014996608 00000100:00100000:1.0:1789762805.453701:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 261 00000100:00100000:1.0:1789762805.453712:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13718:x1876702014996608:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:1.0:1789762805.453737:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762805.453756:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b03f2b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5d0 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.453848:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03f32 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.453865:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03f32 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.453879:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03f32 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762805.453891:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2b:0x0]: returning size 104857600 00010000:00010000:1.0:1789762805.454003:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b03f2b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x29:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5d0 expref: 13 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.454022:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03f32 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5d0 expref: 12 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.454033:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff981ea0aa3c00) 00010000:00010000:1.0:1789762805.454037:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:1.0:1789762805.454040:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81482000, rc 0) 00000100:00100000:1.0:1789762805.454129:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:13718:x1876702014996608:12345-192.168.203.48@tcp:101:dd.500 Request processed in 418us (567us total) trans 4294967515 rc 0/0 00000100:00100000:1.0:1789762805.454140:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 261 00000100:00100000:2.0:1789762805.455449:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762805.455495:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014996736 00000100:00100000:3.0:1789762805.455500:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014996736 00000100:00100000:3.0:1789762805.455531:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 166 00000100:00100000:3.0:1789762805.455537:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702014996736:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762805.455546:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762805.455558:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b03f40 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5d7 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762805.455577:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 104857600, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x3200b 00010000:00010000:3.0:1789762805.455588:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=328s v=(0 0) 00010000:00010000:3.0:1789762805.455592:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762805.455598:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b03f40 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5d7 expref: 6 pid: 6647 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.455608:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b03f40 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5d7 expref: 6 pid: 6647 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762805.455616:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb5200, rc 0) 00000100:00100000:3.0:1789762805.455642:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014996736:12345-192.168.203.48@tcp:101:dd.500 Request processed in 106us (195us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762805.455648:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 166 00000100:00100000:0.0:1789762805.458269:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762805.458314:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014996864 00000100:00100000:2.0:1789762805.458333:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014996864 00000100:00100000:2.0:1789762805.458368:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268420047115 time 268420123087 state 0x0 ref 1 00000100:00100000:2.0:1789762805.458375:0:14913:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 6: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e87804400 gen 0 rate 10 depth 3 token 0 deadline 268420047115 time 268420123087 state 0x0 ref 1 00000100:00100000:2.0:1789762805.458395:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 3 deadline 328812302342 time 328762302342 state 0x1 ref 1 00000100:00100000:2.0:1789762805.458415:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 328812302342 time 328762324500 state 0x0 ref 1 00000100:00100000:2.0:1789762805.458423:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014996864:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762805.458447:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03f47 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.458456:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762805.458459:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762805.458463:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03f47 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.458471:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03f47 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.462200:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03f47 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.462211:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03f47 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.462224:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03f47 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.462230:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762805.462234:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03f47 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.462241:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03f47 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.462253:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03f47 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.462259:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b03f47 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762805.462300:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014996864:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3876us (4030us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762805.462306:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 593 00000100:00100000:0.0:1789762805.462313:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 328812302342 time 328762324500 state 0x2 ref 0 00000100:00100000:0.0:1789762805.464236:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762805.464276:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014996992 00000100:00100000:2.0:1789762805.464281:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014996992 00000100:00100000:2.0:1789762805.464303:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 328812302342 time 328762324500 state 0x0 ref 1 00000100:00100000:2.0:1789762805.464311:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 2 deadline 328812324500 time 328762324500 state 0x1 ref 1 00000100:00100000:2.0:1789762805.464342:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 328812324500 time 328768250466 state 0x0 ref 1 00000100:00100000:2.0:1789762805.464351:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014996992:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762805.464367:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03f4e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.464375:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762805.464378:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762805.464381:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03f4e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.464389:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03f4e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.467010:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03f4e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.467019:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03f4e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.467034:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03f4e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.467039:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762805.467041:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03f4e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.467047:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03f4e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.467058:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03f4e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.467067:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b03f4e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762805.467141:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702014996992:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2790us (2903us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762805.467149:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 594 00000100:00100000:2.0:1789762805.467155:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 328812324500 time 328768250466 state 0x2 ref 0 00000100:00100000:3.0:1789762805.468776:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762805.468812:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014997120 00000100:00100000:0.0:1789762805.468817:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014997120 00000100:00100000:0.0:1789762805.468853:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 328812324500 time 328768250466 state 0x0 ref 1 00000100:00100000:0.0:1789762805.468877:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 1 deadline 328818250466 time 328768250466 state 0x1 ref 1 00000100:00100000:0.0:1789762805.468901:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328818250466 time 328772810430 state 0x0 ref 1 00000100:00100000:0.0:1789762805.468910:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014997120:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762805.468925:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000046748614/0x416193e012b03f55 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.468933:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762805.468936:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762805.468939:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000046748614/0x416193e012b03f55 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.468947:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000046748614/0x416193e012b03f55 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.471255:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000046748614/0x416193e012b03f55 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.471265:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000046748614/0x416193e012b03f55 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.471278:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000046748614/0x416193e012b03f55 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.471283:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa32e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762805.471286:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000046748614/0x416193e012b03f55 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.471292:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000046748614/0x416193e012b03f55 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.471302:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000046748614/0x416193e012b03f55 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.471316:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000046748614/0x416193e012b03f55 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762805.471372:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014997120:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2460us (2594us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762805.471381:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 595 00000100:00100000:2.0:1789762805.471391:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328818250466 time 328772810430 state 0x2 ref 0 00000100:00100000:0.0:1789762805.473384:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.473432:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014997248 00000100:00100000:1.0:1789762805.473438:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014997248 00000100:00100000:1.0:1789762805.473468:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328818250466 time 328772810430 state 0x0 ref 1 00000100:00100000:1.0:1789762805.473480:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328822810430 time 328772810430 state 0x1 ref 1 00000100:00100000:1.0:1789762805.518991:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328822810430 time 328822882440 state 0x0 ref 1 00000100:00100000:1.0:1789762805.519015:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014997248:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762805.519055:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f5c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.519072:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.519077:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.519083:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f5c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.519122:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f5c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.522337:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f5c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.522350:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f5c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.522360:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f5c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.522368:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.522372:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f5c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.522381:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f5c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.522398:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f5c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.522406:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f5c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.522460:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014997248:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3451us (49075us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762805.522469:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 596 00000100:00100000:1.0:1789762805.522479:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328822810430 time 328822882440 state 0x2 ref 0 00000100:00100000:2.0:1789762805.524506:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.524546:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014997376 00000100:00100000:1.0:1789762805.524554:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014997376 00000100:00100000:1.0:1789762805.524596:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328822810430 time 328822882440 state 0x0 ref 1 00000100:00100000:1.0:1789762805.524610:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328872882440 time 328822882440 state 0x1 ref 1 00000100:00100000:1.0:1789762805.569037:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328872882440 time 328872935176 state 0x0 ref 1 00000100:00100000:1.0:1789762805.569055:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014997376:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762805.569101:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f63 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.569114:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.569118:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.569122:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f63 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.569132:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f63 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.572513:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f63 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.572527:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f63 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.572538:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f63 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.572547:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.572553:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f63 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.572564:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f63 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.572583:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f63 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.572592:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f63 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.572656:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014997376:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3602us (48149us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762805.572668:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 597 00000100:00100000:1.0:1789762805.572679:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328872882440 time 328872935176 state 0x2 ref 0 00000100:00100000:2.0:1789762805.574546:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.574594:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014997504 00000100:00100000:1.0:1789762805.574603:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014997504 00000100:00100000:1.0:1789762805.574645:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328872882440 time 328872935176 state 0x0 ref 1 00000100:00100000:1.0:1789762805.574666:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328922935176 time 328872935176 state 0x1 ref 1 00000100:00100000:2.0:1789762805.619144:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328922935176 time 328923040307 state 0x0 ref 1 00000100:00100000:2.0:1789762805.619180:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014997504:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762805.619215:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b03f6a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.619229:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762805.619234:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762805.619240:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b03f6a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.619255:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b03f6a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.622507:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b03f6a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.622518:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b03f6a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.622525:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b03f6a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.622541:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762805.622545:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b03f6a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.622553:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b03f6a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.622568:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b03f6a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.622574:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b03f6a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762805.622627:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014997504:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3452us (48082us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762805.622636:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 598 00000100:00100000:0.0:1789762805.622643:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328922935176 time 328923040307 state 0x2 ref 0 00000100:00100000:0.0:1789762805.624693:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.624745:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014997632 00000100:00100000:1.0:1789762805.624754:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014997632 00000100:00100000:1.0:1789762805.624796:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328922935176 time 328923040307 state 0x0 ref 1 00000100:00100000:1.0:1789762805.624810:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328973040307 time 328923040307 state 0x1 ref 1 00000100:00100000:1.0:1789762805.669203:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328973040307 time 328973098950 state 0x0 ref 1 00000100:00100000:1.0:1789762805.669224:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014997632:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762805.669267:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f71 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.669280:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.669284:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.669288:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f71 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.669299:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f71 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.672463:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f71 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.672483:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f71 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.672494:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f71 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.672501:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.672505:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f71 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.672512:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f71 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.672527:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f71 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.672533:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f71 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.672581:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014997632:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3363us (47889us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762805.672588:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 599 00000100:00100000:1.0:1789762805.672595:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328973040307 time 328973098950 state 0x2 ref 0 00000100:00100000:2.0:1789762805.674583:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.674617:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014997760 00000100:00100000:1.0:1789762805.674625:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014997760 00000100:00100000:1.0:1789762805.674657:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328973040307 time 328973098950 state 0x0 ref 1 00000100:00100000:1.0:1789762805.674668:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329023098950 time 328973098950 state 0x1 ref 1 00010000:00010000:0.0:1789762805.687166:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762805.719252:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329023098950 time 329023147777 state 0x0 ref 1 00000100:00100000:1.0:1789762805.719272:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014997760:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762805.719306:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f78 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.719335:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762805.719340:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.719347:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f78 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.719365:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f78 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.722702:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f78 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.722713:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f78 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.722720:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f78 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.722727:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.722731:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f78 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.722738:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f78 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.722752:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f78 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.722757:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f78 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.722811:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014997760:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3543us (48227us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762805.722822:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 600 00000100:00100000:1.0:1789762805.722832:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329023098950 time 329023147777 state 0x2 ref 0 00000100:00100000:2.0:1789762805.724611:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.724650:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014997888 00000100:00100000:1.0:1789762805.724658:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014997888 00000100:00100000:1.0:1789762805.724694:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329023098950 time 329023147777 state 0x0 ref 1 00000100:00100000:1.0:1789762805.724705:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329073147777 time 329023147777 state 0x1 ref 1 00000100:00100000:0.0:1789762805.769391:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329073147777 time 329073288693 state 0x0 ref 1 00000100:00100000:0.0:1789762805.769411:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014997888:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762805.769456:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2e1e491/0x416193e012b03f7f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.769482:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762805.769486:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762805.769490:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2e1e491/0x416193e012b03f7f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.769501:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2e1e491/0x416193e012b03f7f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.772679:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2e1e491/0x416193e012b03f7f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.772690:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2e1e491/0x416193e012b03f7f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.772697:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2e1e491/0x416193e012b03f7f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.772702:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e879da000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762805.772706:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2e1e491/0x416193e012b03f7f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.772712:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2e1e491/0x416193e012b03f7f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.772726:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2e1e491/0x416193e012b03f7f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.772742:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2e1e491/0x416193e012b03f7f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762805.772799:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014997888:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3393us (48187us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762805.772809:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 601 00000100:00100000:0.0:1789762805.772818:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329073147777 time 329073288693 state 0x2 ref 0 00000100:00100000:0.0:1789762805.774615:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.774662:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014998016 00000100:00100000:1.0:1789762805.774670:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014998016 00000100:00100000:1.0:1789762805.774713:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329073147777 time 329073288693 state 0x0 ref 1 00000100:00100000:1.0:1789762805.774724:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329123288693 time 329073288693 state 0x1 ref 1 00000100:00100000:1.0:1789762805.819451:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329123288693 time 329123343896 state 0x0 ref 1 00000100:00100000:1.0:1789762805.819478:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014998016:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762805.819516:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f86 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.819529:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.819533:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.819537:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f86 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.819548:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f86 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.822480:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f86 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.822489:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f86 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.822496:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f86 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.822503:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.822507:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f86 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.822514:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f86 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.822529:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f86 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.822535:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f86 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.822582:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014998016:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3114us (47968us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762805.822589:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 602 00000100:00100000:1.0:1789762805.822596:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329123288693 time 329123343896 state 0x2 ref 0 00000100:00100000:2.0:1789762805.824337:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.824373:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014998144 00000100:00100000:1.0:1789762805.824381:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014998144 00000100:00100000:1.0:1789762805.824412:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329123288693 time 329123343896 state 0x0 ref 1 00000100:00100000:1.0:1789762805.824422:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329173343896 time 329123343896 state 0x1 ref 1 00000100:00100000:1.0:1789762805.869501:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329173343896 time 329173397463 state 0x0 ref 1 00000100:00100000:1.0:1789762805.869522:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014998144:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762805.869559:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f8d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.869572:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.869576:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.869581:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f8d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.869591:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f8d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.872440:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f8d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.872455:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f8d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.872465:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f8d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.872484:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.872489:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f8d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.872503:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f8d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.872518:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f8d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.872525:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f8d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.872585:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014998144:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3067us (48248us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762805.872592:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 603 00000100:00100000:1.0:1789762805.872600:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329173343896 time 329173397463 state 0x2 ref 0 00000100:00100000:0.0:1789762805.874535:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.874574:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014998272 00000100:00100000:1.0:1789762805.874582:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014998272 00000100:00100000:1.0:1789762805.874614:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329173343896 time 329173397463 state 0x0 ref 1 00000100:00100000:1.0:1789762805.874625:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329223397463 time 329173397463 state 0x1 ref 1 00000100:00100000:0.0:1789762805.919573:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329223397463 time 329223469594 state 0x0 ref 1 00000100:00100000:0.0:1789762805.919599:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014998272:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762805.919635:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b03f94 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.919648:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762805.919652:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762805.919657:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b03f94 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762805.919668:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b03f94 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.923263:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b03f94 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.923283:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b03f94 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.923294:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b03f94 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.923303:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762805.923308:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b03f94 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.923336:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b03f94 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.923357:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b03f94 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762805.923366:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b03f94 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762805.923443:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014998272:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3851us (48903us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762805.923454:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 604 00000100:00100000:2.0:1789762805.923468:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329223397463 time 329223469594 state 0x2 ref 0 00000100:00100000:2.0:1789762805.925651:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.925695:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014998400 00000100:00100000:1.0:1789762805.925705:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014998400 00000100:00100000:1.0:1789762805.925751:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329223397463 time 329223469594 state 0x0 ref 1 00000100:00100000:1.0:1789762805.925767:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329273469594 time 329223469594 state 0x1 ref 1 00000100:00100000:1.0:1789762805.969632:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329273469594 time 329273529045 state 0x0 ref 1 00000100:00100000:1.0:1789762805.969652:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014998400:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762805.969687:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f9b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.969701:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.969711:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762805.969716:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f9b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.969728:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f9b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.973304:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f9b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.973333:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f9b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.973342:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f9b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.973347:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762805.973351:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f9b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.973357:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f9b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.973373:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f9b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762805.973379:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03f9b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762805.973432:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014998400:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3784us (47782us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762805.973441:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 605 00000100:00100000:1.0:1789762805.973452:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329273469594 time 329273529045 state 0x2 ref 0 00000100:00100000:3.0:1789762805.975822:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762805.975870:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014998528 00000100:00100000:1.0:1789762805.975880:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014998528 00000100:00100000:1.0:1789762805.975924:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329273469594 time 329273529045 state 0x0 ref 1 00000100:00100000:1.0:1789762805.975940:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329323529045 time 329273529045 state 0x1 ref 1 00000100:00100000:1.0:1789762806.019702:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329323529045 time 329323594091 state 0x0 ref 1 00000100:00100000:1.0:1789762806.019736:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014998528:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.019775:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fa2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.019793:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.019799:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.019806:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fa2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.019830:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fa2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.022998:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fa2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.023011:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fa2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.023021:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fa2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.023030:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.023035:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fa2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.023046:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fa2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.023064:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fa2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.023073:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fa2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.023165:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014998528:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3435us (47343us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.023176:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 606 00000100:00100000:1.0:1789762806.023186:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329323529045 time 329323594091 state 0x2 ref 0 00000100:00100000:3.0:1789762806.025413:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.025455:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014998656 00000100:00100000:1.0:1789762806.025464:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014998656 00000100:00100000:1.0:1789762806.025507:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329323529045 time 329323594091 state 0x0 ref 1 00000100:00100000:1.0:1789762806.025523:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329373594091 time 329323594091 state 0x1 ref 1 00000100:00100000:2.0:1789762806.069825:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329373594091 time 329373720147 state 0x0 ref 1 00000100:00100000:2.0:1789762806.069846:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014998656:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762806.069880:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a52f85e7/0x416193e012b03fa9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.069894:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762806.069898:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762806.069902:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a52f85e7/0x416193e012b03fa9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.069927:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a52f85e7/0x416193e012b03fa9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.072939:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a52f85e7/0x416193e012b03fa9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.072950:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a52f85e7/0x416193e012b03fa9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.072957:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a52f85e7/0x416193e012b03fa9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.072963:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc4a3a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762806.072974:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a52f85e7/0x416193e012b03fa9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.072980:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a52f85e7/0x416193e012b03fa9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.072993:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a52f85e7/0x416193e012b03fa9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.072999:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a52f85e7/0x416193e012b03fa9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762806.073047:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014998656:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3208us (47635us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762806.073054:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 607 00000100:00100000:0.0:1789762806.073069:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329373594091 time 329373720147 state 0x2 ref 0 00000100:00100000:2.0:1789762806.075042:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.075108:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014998784 00000100:00100000:1.0:1789762806.075117:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014998784 00000100:00100000:1.0:1789762806.075153:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329373594091 time 329373720147 state 0x0 ref 1 00000100:00100000:1.0:1789762806.075164:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329423720147 time 329373720147 state 0x1 ref 1 00000100:00100000:1.0:1789762806.119901:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329423720147 time 329423793592 state 0x0 ref 1 00000100:00100000:1.0:1789762806.119926:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014998784:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.119967:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.119985:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.119991:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.119997:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.120011:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.123404:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.123418:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.123428:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.123438:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.123444:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.123454:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.123474:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.123483:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.123558:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014998784:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3638us (48514us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.123570:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 608 00000100:00100000:1.0:1789762806.123582:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329423720147 time 329423793592 state 0x2 ref 0 00000100:00100000:2.0:1789762806.125680:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.125732:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014998912 00000100:00100000:1.0:1789762806.125741:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014998912 00000100:00100000:1.0:1789762806.125808:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329423720147 time 329423793592 state 0x0 ref 1 00000100:00100000:1.0:1789762806.125823:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329473793592 time 329423793592 state 0x1 ref 1 00000100:00100000:1.0:1789762806.169954:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329473793592 time 329473850487 state 0x0 ref 1 00000100:00100000:1.0:1789762806.169976:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014998912:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.170018:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.170032:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.170036:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.170041:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.170052:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.173466:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.173482:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.173492:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.173501:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.173504:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.173511:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.173526:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.173534:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fb7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.173592:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014998912:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3623us (47913us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.173600:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 609 00000100:00100000:1.0:1789762806.173607:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329473793592 time 329473850487 state 0x2 ref 0 00000100:00100000:2.0:1789762806.175786:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.175830:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014999040 00000100:00100000:1.0:1789762806.175838:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014999040 00000100:00100000:1.0:1789762806.175881:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329473793592 time 329473850487 state 0x0 ref 1 00000100:00100000:1.0:1789762806.175897:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329523850487 time 329473850487 state 0x1 ref 1 00000100:00100000:0.0:1789762806.220075:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329523850487 time 329523967686 state 0x0 ref 1 00000100:00100000:0.0:1789762806.220129:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014999040:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762806.220165:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fbe lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.220179:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762806.220183:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762806.220188:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fbe lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.220200:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fbe lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.223605:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fbe lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.223617:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fbe lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.223624:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fbe lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.223629:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762806.223643:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fbe lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.223650:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fbe lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.223667:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fbe lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.223673:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fbe lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762806.223722:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014999040:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3599us (47934us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762806.223732:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 610 00000100:00100000:0.0:1789762806.223742:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329523850487 time 329523967686 state 0x2 ref 0 00000100:00100000:3.0:1789762806.225803:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.225867:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014999168 00000100:00100000:1.0:1789762806.225877:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014999168 00000100:00100000:1.0:1789762806.225923:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329523850487 time 329523967686 state 0x0 ref 1 00000100:00100000:1.0:1789762806.225939:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329573967686 time 329523967686 state 0x1 ref 1 00000100:00100000:1.0:1789762806.270141:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329573967686 time 329574035345 state 0x0 ref 1 00000100:00100000:1.0:1789762806.270165:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014999168:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.270203:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fc5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.270219:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.270224:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.270230:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fc5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.270244:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fc5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.273503:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fc5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.273516:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fc5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.273526:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fc5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.273535:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.273540:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fc5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.273549:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fc5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.273567:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fc5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.273576:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fc5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.273633:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014999168:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3474us (47832us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.273643:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 611 00000100:00100000:1.0:1789762806.273655:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329573967686 time 329574035345 state 0x2 ref 0 00000100:00100000:3.0:1789762806.275580:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.275627:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014999296 00000100:00100000:1.0:1789762806.275636:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014999296 00000100:00100000:1.0:1789762806.275673:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329573967686 time 329574035345 state 0x0 ref 1 00000100:00100000:1.0:1789762806.275688:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329624035345 time 329574035345 state 0x1 ref 1 00000100:00100000:1.0:1789762806.320204:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329624035345 time 329624101229 state 0x0 ref 1 00000100:00100000:1.0:1789762806.320225:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014999296:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.320260:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fcc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.320273:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.320276:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.320281:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fcc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.320291:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fcc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.323169:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fcc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.323180:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fcc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.323189:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fcc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.323195:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.323199:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fcc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.323205:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fcc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.323219:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fcc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.323225:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b03fcc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.323273:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014999296:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3054us (47693us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.323280:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 612 00000100:00100000:1.0:1789762806.323287:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329624035345 time 329624101229 state 0x2 ref 0 00000100:00100000:2.0:1789762806.325196:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.325246:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014999424 00000100:00100000:1.0:1789762806.325256:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014999424 00000100:00100000:1.0:1789762806.325296:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329624035345 time 329624101229 state 0x0 ref 1 00000100:00100000:1.0:1789762806.325329:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329674101229 time 329624101229 state 0x1 ref 1 00000100:00100000:0.0:1789762806.370334:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329674101229 time 329674217845 state 0x0 ref 1 00000100:00100000:0.0:1789762806.370355:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014999424:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762806.370389:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fd3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.370402:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762806.370406:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762806.370410:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fd3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.370434:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fd3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.373159:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fd3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.373173:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fd3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.373184:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fd3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.373193:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762806.373199:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fd3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.373209:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fd3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.373228:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fd3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.373234:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fd3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762806.373292:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014999424:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2943us (48096us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762806.373299:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 613 00000100:00100000:0.0:1789762806.373307:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329674101229 time 329674217845 state 0x2 ref 0 00000100:00100000:0.0:1789762806.375215:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.375271:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014999552 00000100:00100000:1.0:1789762806.375281:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014999552 00000100:00100000:1.0:1789762806.375337:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329674101229 time 329674217845 state 0x0 ref 1 00000100:00100000:1.0:1789762806.375353:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329724217845 time 329674217845 state 0x1 ref 1 00000001:00080000:2.0:1789762806.398034:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967515 is committed 00000004:00080000:2.0:1789762806.398045:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 17 committed 00000004:00080000:2.0:1789762806.398055:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 20 committed 00000100:00100000:2.0:1789762806.398154:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc554000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022166528:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762806.398208:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762806.398241:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022166528 00000100:00100000:0.0:1789762806.398248:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022166528 00000100:00100000:2.0:1789762806.398265:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789762806.398276:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdcb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022166400:0@lo:6:osp-syn-1-0.0 00000100:00080000:1.0:1789762806.398280:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762806.398324:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:2.0:1789762806.398330:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762806.398340:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022166528:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789762806.398343:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022166400 00000100:00100000:1.0:1789762806.398349:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022166400 00002000:00080000:0.0:1789762806.398368:0:6881:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:16 count 1 00000100:00100000:1.0:1789762806.398386:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:1.0:1789762806.398395:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022166400:12345-0@lo:6:osp-syn-1-0.0 00002000:00010000:0.0:1789762806.398402:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x10:0x0] initial LVB size: 8388608, mtime: 0x6aad9cef, atime: 0x0, ctime: 0x6aad9cef, blocks: 0x4007 00002000:00080000:1.0:1789762806.398409:0:10916:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:11 count 1 00010000:00010000:0.0:1789762806.398412:0:6881:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762806.398427:0:10916:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xb:0x0] initial LVB size: 8388608, mtime: 0x6aad9cef, atime: 0x0, ctime: 0x6aad9cef, blocks: 0x4007 00010000:00010000:0.0:1789762806.398430:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=329s v=(0 0) 00010000:00010000:1.0:1789762806.398435:0:10916:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b03fe1 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.398437:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762806.398450:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.398452:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=329s v=(0 0) 00010000:00010000:1.0:1789762806.398463:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xb:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.398471:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b03fe1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.398473:0:6881:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.398497:0:6881:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.398498:0:10916:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b03fe1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.398525:0:10916:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b03fe1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.398544:0:10916:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b03fe1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.398549:0:6881:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.398564:0:10916:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1789762806.398575:0:6881:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:2.0:1789762806.398596:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1789762806.398596:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762806.398606:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b03fe1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762806.398610:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762806.398633:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762806.398634:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762806.398638:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.398642:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b03fe1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762806.398654:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.398662:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b03fe1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762806.398670:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fda lrc: 1/0,0 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762806.398678:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b03fda lrc: 0/0,0 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.398712:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b03fe1 lrc: 1/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762806.398712:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762806.398717:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762806.398723:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762806.398725:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b03fe1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xb:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.398751:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762806.398753:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762806.398757:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762806.399526:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022166400:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1130us (1214us total) trans 4294967398 rc 0/0 00000100:00100000:0.0:1789762806.399534:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022166528:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1194us (1321us total) trans 4294967910 rc 0/0 00000100:00100000:1.0:1789762806.399535:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:0.0:1789762806.399549:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00000100:00100000:2.0:1789762806.399553:0:3274:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967397 gen 2 00000004:00080000:2.0:1789762806.399559:0:3274:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981d846f0a80, transno 4294967397 00000100:00080000:1.0:1789762806.399564:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762806.399575:0:3274:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff981ebfdcb100 x1876702022166400/t4294967398(4294967398) o6->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789762822 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1789762806.399584:0:3274:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebfdcb100/2, rc 0, transno 4294967398 00000100:00100000:2.0:1789762806.399590:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdcb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022166400:0@lo:6:osp-syn-1-0.0 00000004:00080000:2.0:1789762806.399604:0:3274:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebc554000/2, rc 0, transno 4294967910 00000100:00100000:2.0:1789762806.399608:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc554000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022166528:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789762806.420385:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329724217845 time 329724281838 state 0x0 ref 1 00000100:00100000:1.0:1789762806.420404:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014999552:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.420436:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fe8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.420449:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.420452:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.420457:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fe8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.420467:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fe8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.422928:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fe8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.422939:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fe8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.422949:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fe8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.422958:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.422963:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fe8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.422973:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fe8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.422990:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fe8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.422996:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fe8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.423046:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014999552:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2647us (47832us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.423052:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 614 00000100:00100000:1.0:1789762806.423060:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329724217845 time 329724281838 state 0x2 ref 0 00000100:00100000:3.0:1789762806.425182:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.425215:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014999680 00000100:00100000:1.0:1789762806.425224:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014999680 00000100:00100000:1.0:1789762806.425258:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329724217845 time 329724281838 state 0x0 ref 1 00000100:00100000:1.0:1789762806.425269:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329774281838 time 329724281838 state 0x1 ref 1 00000100:00100000:1.0:1789762806.470452:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329774281838 time 329774344852 state 0x0 ref 1 00000100:00100000:1.0:1789762806.470479:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014999680:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.470528:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fef lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.470546:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.470551:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.470557:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fef lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.470572:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fef lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.473570:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fef lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.473583:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fef lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.473594:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.473603:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.473609:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.473620:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.473639:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fef lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.473647:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03fef lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.473707:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014999680:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3235us (48524us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.473717:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 615 00000100:00100000:1.0:1789762806.473727:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329774281838 time 329774344852 state 0x2 ref 0 00000100:00100000:3.0:1789762806.475559:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.475606:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014999808 00000100:00100000:1.0:1789762806.475614:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014999808 00000100:00100000:1.0:1789762806.475656:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329774281838 time 329774344852 state 0x0 ref 1 00000100:00100000:1.0:1789762806.475674:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329824344852 time 329774344852 state 0x1 ref 1 00000100:00100000:0.0:1789762806.520536:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329824344852 time 329824432721 state 0x0 ref 1 00000100:00100000:0.0:1789762806.520557:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014999808:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762806.520592:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b5928a14/0x416193e012b03ff6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.520606:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762806.520610:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762806.520615:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b5928a14/0x416193e012b03ff6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.520626:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b5928a14/0x416193e012b03ff6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.523518:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b5928a14/0x416193e012b03ff6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.523529:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b5928a14/0x416193e012b03ff6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.523536:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b5928a14/0x416193e012b03ff6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.523556:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebff42200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762806.523561:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5928a14/0x416193e012b03ff6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.523567:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b5928a14/0x416193e012b03ff6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.523582:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b5928a14/0x416193e012b03ff6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.523587:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b5928a14/0x416193e012b03ff6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762806.523634:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702014999808:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3083us (48077us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762806.523642:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 616 00000100:00100000:2.0:1789762806.523651:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329824344852 time 329824432721 state 0x2 ref 0 00000100:00100000:0.0:1789762806.525562:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.525642:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702014999936 00000100:00100000:1.0:1789762806.525651:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702014999936 00000100:00100000:1.0:1789762806.525688:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329824344852 time 329824432721 state 0x0 ref 1 00000100:00100000:1.0:1789762806.525699:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329874432721 time 329824432721 state 0x1 ref 1 00000100:00100000:1.0:1789762806.570597:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329874432721 time 329874493962 state 0x0 ref 1 00000100:00100000:1.0:1789762806.570616:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014999936:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.570649:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03ffd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.570662:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.570666:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.570670:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03ffd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.570680:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03ffd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.573706:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03ffd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.573729:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03ffd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.573741:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03ffd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.573750:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.573755:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03ffd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.573765:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03ffd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.573783:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03ffd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.573792:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b03ffd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.573854:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702014999936:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3240us (48291us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.573865:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 617 00000100:00100000:1.0:1789762806.573877:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329874432721 time 329874493962 state 0x2 ref 0 00000100:00100000:0.0:1789762806.575708:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.575753:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015000064 00000100:00100000:1.0:1789762806.575763:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015000064 00000100:00100000:1.0:1789762806.575805:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329874432721 time 329874493962 state 0x0 ref 1 00000100:00100000:1.0:1789762806.575819:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329924493962 time 329874493962 state 0x1 ref 1 00000100:00100000:1.0:1789762806.620649:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329924493962 time 329924544609 state 0x0 ref 1 00000100:00100000:1.0:1789762806.620670:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015000064:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.620705:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04004 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.620720:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.620725:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.620731:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04004 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.620744:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04004 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.623535:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04004 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.623547:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04004 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.623557:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04004 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.623567:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.623572:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04004 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.623583:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04004 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.623600:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04004 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.623608:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04004 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.623665:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015000064:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2999us (47964us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.623676:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 618 00000100:00100000:1.0:1789762806.623686:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329924493962 time 329924544609 state 0x2 ref 0 00000100:00100000:3.0:1789762806.625466:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.625508:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015000192 00000100:00100000:1.0:1789762806.625517:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015000192 00000100:00100000:1.0:1789762806.625554:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329924493962 time 329924544609 state 0x0 ref 1 00000100:00100000:1.0:1789762806.625569:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329974544609 time 329924544609 state 0x1 ref 1 00000100:00100000:2.0:1789762806.670729:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329974544609 time 329974627697 state 0x0 ref 1 00000100:00100000:2.0:1789762806.670746:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015000192:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762806.670775:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1c5f427/0x416193e012b0400b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.670804:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762806.670807:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762806.670818:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1c5f427/0x416193e012b0400b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.670828:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1c5f427/0x416193e012b0400b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.673617:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1c5f427/0x416193e012b0400b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.673639:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c1c5f427/0x416193e012b0400b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.673649:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c1c5f427/0x416193e012b0400b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.673658:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c6fe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762806.673663:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1c5f427/0x416193e012b0400b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.673683:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1c5f427/0x416193e012b0400b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.673705:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c1c5f427/0x416193e012b0400b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.673715:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1c5f427/0x416193e012b0400b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762806.673767:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015000192:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3025us (48301us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762806.673774:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 619 00000100:00100000:0.0:1789762806.673783:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329974544609 time 329974627697 state 0x2 ref 0 00000100:00100000:3.0:1789762806.675620:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.675664:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015000320 00000100:00100000:1.0:1789762806.675672:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015000320 00000100:00100000:1.0:1789762806.675707:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 329974544609 time 329974627697 state 0x0 ref 1 00000100:00100000:1.0:1789762806.675717:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330024627697 time 329974627697 state 0x1 ref 1 00010000:00010000:1.0:1789762806.711194:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762806.720798:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330024627697 time 330024693028 state 0x0 ref 1 00000100:00100000:1.0:1789762806.720823:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015000320:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.720859:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04012 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.720874:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762806.720878:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.720882:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04012 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.720892:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04012 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.724104:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04012 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.724117:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04012 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.724124:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04012 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.724132:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.724138:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04012 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.724149:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04012 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.724169:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04012 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.724178:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04012 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.724242:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015000320:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3424us (48620us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.724253:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 620 00000100:00100000:1.0:1789762806.724264:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330024627697 time 330024693028 state 0x2 ref 0 00000100:00100000:3.0:1789762806.726176:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.726207:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015000448 00000100:00100000:1.0:1789762806.726216:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015000448 00000100:00100000:1.0:1789762806.726252:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330024627697 time 330024693028 state 0x0 ref 1 00000100:00100000:1.0:1789762806.726263:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330074693028 time 330024693028 state 0x1 ref 1 00000100:00100000:1.0:1789762806.770853:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330074693028 time 330074749442 state 0x0 ref 1 00000100:00100000:1.0:1789762806.770873:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015000448:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.770908:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04019 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.770922:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.770926:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.770931:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04019 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.770942:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04019 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.773533:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04019 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.773546:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04019 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.773553:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04019 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.773559:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.773562:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04019 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.773568:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04019 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.773581:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04019 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.773587:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04019 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.773647:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015000448:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2780us (47471us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.773654:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 621 00000100:00100000:1.0:1789762806.773662:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330074693028 time 330074749442 state 0x2 ref 0 00000100:00100000:3.0:1789762806.775535:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.775582:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015000576 00000100:00100000:1.0:1789762806.775590:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015000576 00000100:00100000:1.0:1789762806.775624:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330074693028 time 330074749442 state 0x0 ref 1 00000100:00100000:1.0:1789762806.775638:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330124749442 time 330074749442 state 0x1 ref 1 00000100:00100000:0.0:1789762806.820937:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330124749442 time 330124837633 state 0x0 ref 1 00000100:00100000:0.0:1789762806.820966:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015000576:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762806.820994:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a3d9a941/0x416193e012b04020 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.821007:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762806.821011:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762806.821015:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3d9a941/0x416193e012b04020 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.821025:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3d9a941/0x416193e012b04020 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.823670:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a3d9a941/0x416193e012b04020 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.823679:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a3d9a941/0x416193e012b04020 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.823686:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a3d9a941/0x416193e012b04020 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.823691:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc4a3800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762806.823695:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3d9a941/0x416193e012b04020 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.823702:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a3d9a941/0x416193e012b04020 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.823721:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a3d9a941/0x416193e012b04020 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.823726:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a3d9a941/0x416193e012b04020 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762806.823769:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015000576:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2808us (48235us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762806.823775:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 622 00000100:00100000:0.0:1789762806.823782:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330124749442 time 330124837633 state 0x2 ref 0 00000100:00100000:2.0:1789762806.825508:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.825552:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015000704 00000100:00100000:1.0:1789762806.825560:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015000704 00000100:00100000:1.0:1789762806.825591:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330124749442 time 330124837633 state 0x0 ref 1 00000100:00100000:1.0:1789762806.825601:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330174837633 time 330124837633 state 0x1 ref 1 00000100:00100000:1.0:1789762806.870995:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330174837633 time 330174893414 state 0x0 ref 1 00000100:00100000:1.0:1789762806.871013:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015000704:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.871045:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04027 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.871058:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.871061:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.871066:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04027 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.871076:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04027 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.874480:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04027 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.874500:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04027 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.874510:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04027 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.874519:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.874524:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04027 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.874534:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04027 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.874558:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04027 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.874568:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04027 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.874642:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015000704:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3629us (49130us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.874654:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 623 00000100:00100000:1.0:1789762806.874667:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330174837633 time 330174893414 state 0x2 ref 0 00000100:00100000:2.0:1789762806.878082:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.878154:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015000832 00000100:00100000:1.0:1789762806.878166:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015000832 00000100:00100000:1.0:1789762806.878228:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330174837633 time 330174893414 state 0x0 ref 1 00000100:00100000:1.0:1789762806.878245:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330224893414 time 330174893414 state 0x1 ref 1 00000100:00100000:1.0:1789762806.921046:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330224893414 time 330224943206 state 0x0 ref 1 00000100:00100000:1.0:1789762806.921065:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015000832:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762806.921113:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0402e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.921127:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.921131:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762806.921136:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0402e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.921146:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0402e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.924138:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0402e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.924149:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0402e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.924157:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0402e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.924163:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762806.924167:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0402e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.924174:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0402e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.924188:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0402e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762806.924193:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0402e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762806.924242:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015000832:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3182us (46169us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762806.924249:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 624 00000100:00100000:1.0:1789762806.924256:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330224893414 time 330224943206 state 0x2 ref 0 00000100:00100000:3.0:1789762806.926081:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.926154:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015000960 00000100:00100000:1.0:1789762806.926164:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015000960 00000100:00100000:1.0:1789762806.926200:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330224893414 time 330224943206 state 0x0 ref 1 00000100:00100000:1.0:1789762806.926211:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330274943206 time 330224943206 state 0x1 ref 1 00000100:00100000:0.0:1789762806.971164:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330274943206 time 330275060530 state 0x0 ref 1 00000100:00100000:0.0:1789762806.971186:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015000960:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762806.971221:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bb748ec/0x416193e012b04035 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.971234:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762806.971238:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762806.971242:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bb748ec/0x416193e012b04035 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762806.971254:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bb748ec/0x416193e012b04035 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.974385:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bb748ec/0x416193e012b04035 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.974401:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bb748ec/0x416193e012b04035 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.974412:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bb748ec/0x416193e012b04035 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.974430:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfce00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762806.974436:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bb748ec/0x416193e012b04035 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.974446:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bb748ec/0x416193e012b04035 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.974473:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bb748ec/0x416193e012b04035 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762806.974479:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bb748ec/0x416193e012b04035 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762806.974528:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015000960:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3348us (48448us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762806.974535:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 625 00000100:00100000:2.0:1789762806.974543:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330274943206 time 330275060530 state 0x2 ref 0 00000100:00100000:3.0:1789762806.976454:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762806.976497:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015001088 00000100:00100000:1.0:1789762806.976505:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015001088 00000100:00100000:1.0:1789762806.976550:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330274943206 time 330275060530 state 0x0 ref 1 00000100:00100000:1.0:1789762806.976566:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330325060530 time 330275060530 state 0x1 ref 1 00000100:00100000:1.0:1789762807.021233:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330325060530 time 330325129374 state 0x0 ref 1 00000100:00100000:1.0:1789762807.021270:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015001088:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.021304:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0403c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.021333:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.021338:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.021343:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0403c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762807.021350:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762807.021355:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0403c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.024282:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0403c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.024296:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0403c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.024319:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0403c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.024329:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.024335:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0403c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.024346:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0403c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.024364:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0403c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.024373:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0403c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.024434:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015001088:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3184us (47979us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.024446:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 626 00000100:00100000:1.0:1789762807.024457:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330325060530 time 330325129374 state 0x2 ref 0 00000100:00100000:3.0:1789762807.026333:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.026371:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015001216 00000100:00100000:1.0:1789762807.026380:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015001216 00000100:00100000:1.0:1789762807.026414:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330325060530 time 330325129374 state 0x0 ref 1 00000100:00100000:1.0:1789762807.026424:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330375129374 time 330325129374 state 0x1 ref 1 00000100:00100000:1.0:1789762807.071280:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330375129374 time 330375179080 state 0x0 ref 1 00000100:00100000:1.0:1789762807.071298:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015001216:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.071342:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04043 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.071355:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.071359:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.071363:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04043 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.071374:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04043 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.074488:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04043 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.074498:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04043 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.074504:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04043 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.074509:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.074524:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04043 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.074534:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04043 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.074552:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04043 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.074562:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04043 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.074609:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015001216:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3315us (48276us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.074616:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 627 00000100:00100000:1.0:1789762807.074624:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330375129374 time 330375179080 state 0x2 ref 0 00000100:00100000:0.0:1789762807.076460:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.076494:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015001344 00000100:00100000:1.0:1789762807.076501:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015001344 00000100:00100000:1.0:1789762807.076539:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330375129374 time 330375179080 state 0x0 ref 1 00000100:00100000:1.0:1789762807.076554:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330425179080 time 330375179080 state 0x1 ref 1 00000100:00100000:2.0:1789762807.121368:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330425179080 time 330425264473 state 0x0 ref 1 00000100:00100000:2.0:1789762807.121387:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015001344:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762807.121426:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002d73cb52/0x416193e012b0404a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.121440:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762807.121444:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762807.121449:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002d73cb52/0x416193e012b0404a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.121460:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002d73cb52/0x416193e012b0404a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762807.124409:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002d73cb52/0x416193e012b0404a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762807.124420:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002d73cb52/0x416193e012b0404a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762807.124428:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002d73cb52/0x416193e012b0404a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762807.124434:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762807.124438:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002d73cb52/0x416193e012b0404a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762807.124445:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002d73cb52/0x416193e012b0404a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762807.124459:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002d73cb52/0x416193e012b0404a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762807.124464:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d73cb52/0x416193e012b0404a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762807.124511:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015001344:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3130us (48052us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762807.124528:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 628 00000100:00100000:0.0:1789762807.124536:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330425179080 time 330425264473 state 0x2 ref 0 00000100:00100000:0.0:1789762807.126275:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.126356:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015001472 00000100:00100000:1.0:1789762807.126366:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015001472 00000100:00100000:1.0:1789762807.126409:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330425179080 time 330425264473 state 0x0 ref 1 00000100:00100000:1.0:1789762807.126421:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330475264473 time 330425264473 state 0x1 ref 1 00000100:00100000:1.0:1789762807.171416:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330475264473 time 330475312390 state 0x0 ref 1 00000100:00100000:1.0:1789762807.171437:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015001472:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.171470:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04051 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.171483:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.171486:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.171491:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04051 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.171501:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04051 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.174753:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04051 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.174769:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04051 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.174780:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04051 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.174789:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.174794:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04051 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.174803:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04051 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.174817:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04051 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.174823:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04051 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.174872:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015001472:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3440us (48596us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.174879:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 629 00000100:00100000:1.0:1789762807.174887:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330475264473 time 330475312390 state 0x2 ref 0 00000100:00100000:3.0:1789762807.176786:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.176827:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015001600 00000100:00100000:1.0:1789762807.176838:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015001600 00000100:00100000:1.0:1789762807.176879:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330475264473 time 330475312390 state 0x0 ref 1 00000100:00100000:1.0:1789762807.176890:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330525312390 time 330475312390 state 0x1 ref 1 00000100:00100000:1.0:1789762807.221468:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330525312390 time 330525367093 state 0x0 ref 1 00000100:00100000:1.0:1789762807.221487:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015001600:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.221518:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04058 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.221531:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.221535:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.221539:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04058 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.221549:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04058 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.224565:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04058 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.224580:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04058 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.224592:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04058 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.224601:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.224606:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04058 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.224616:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04058 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.224633:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04058 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.224643:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04058 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.224693:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015001600:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3211us (47908us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.224700:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 630 00000100:00100000:1.0:1789762807.224708:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330525312390 time 330525367093 state 0x2 ref 0 00000100:00100000:3.0:1789762807.226589:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.226622:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015001728 00000100:00100000:1.0:1789762807.226629:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015001728 00000100:00100000:1.0:1789762807.226660:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330525312390 time 330525367093 state 0x0 ref 1 00000100:00100000:1.0:1789762807.226673:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330575367093 time 330525367093 state 0x1 ref 1 00000100:00100000:0.0:1789762807.271559:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330575367093 time 330575454124 state 0x0 ref 1 00000100:00100000:0.0:1789762807.271579:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015001728:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762807.271615:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000056022fa8/0x416193e012b0405f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762807.271629:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762807.271633:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762807.271637:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056022fa8/0x416193e012b0405f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762807.271648:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056022fa8/0x416193e012b0405f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.274800:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000056022fa8/0x416193e012b0405f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.274819:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056022fa8/0x416193e012b0405f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.274826:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056022fa8/0x416193e012b0405f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.274831:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063bc00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762807.274835:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056022fa8/0x416193e012b0405f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.274847:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056022fa8/0x416193e012b0405f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.274863:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056022fa8/0x416193e012b0405f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.274868:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056022fa8/0x416193e012b0405f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762807.274918:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015001728:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3345us (48330us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762807.274926:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 631 00000100:00100000:2.0:1789762807.274941:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330575367093 time 330575454124 state 0x2 ref 0 00000100:00100000:3.0:1789762807.276883:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.276933:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015001856 00000100:00100000:1.0:1789762807.276943:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015001856 00000100:00100000:1.0:1789762807.276995:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330575367093 time 330575454124 state 0x0 ref 1 00000100:00100000:1.0:1789762807.277007:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330625454124 time 330575454124 state 0x1 ref 1 00000100:00100000:1.0:1789762807.321619:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330625454124 time 330625511426 state 0x0 ref 1 00000100:00100000:1.0:1789762807.321649:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015001856:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.321691:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04066 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.321705:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.321709:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.321714:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04066 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.321725:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04066 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.324908:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04066 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.324929:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04066 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.324941:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04066 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.324950:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.324956:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04066 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.324966:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04066 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.324984:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04066 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.324992:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04066 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.325041:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015001856:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3403us (48159us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.325048:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 632 00000100:00100000:1.0:1789762807.325066:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330625454124 time 330625511426 state 0x2 ref 0 00000100:00100000:0.0:1789762807.327398:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.327437:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015001984 00000100:00100000:1.0:1789762807.327445:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015001984 00000100:00100000:1.0:1789762807.327481:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330625454124 time 330625511426 state 0x0 ref 1 00000100:00100000:1.0:1789762807.327491:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330675511426 time 330625511426 state 0x1 ref 1 00000100:00100000:1.0:1789762807.371684:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330675511426 time 330675576070 state 0x0 ref 1 00000100:00100000:1.0:1789762807.371713:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015001984:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.371756:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0406d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.371775:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.371780:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.371786:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0406d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.371802:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0406d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.375401:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0406d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.375412:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0406d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.375421:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0406d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.375430:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.375435:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0406d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.375445:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0406d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.375463:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0406d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.375472:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0406d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.375532:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015001984:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3829us (48136us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.375539:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 633 00000100:00100000:1.0:1789762807.375547:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330675511426 time 330675576070 state 0x2 ref 0 00000100:00100000:0.0:1789762807.378013:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.378056:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015002112 00000100:00100000:1.0:1789762807.378063:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015002112 00000100:00100000:1.0:1789762807.378115:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330675511426 time 330675576070 state 0x0 ref 1 00000100:00100000:1.0:1789762807.378126:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330725576070 time 330675576070 state 0x1 ref 1 00000100:00100000:2.0:1789762807.421749:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330725576070 time 330725645948 state 0x0 ref 1 00000100:00100000:2.0:1789762807.421777:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015002112:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762807.421817:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b54a1e22/0x416193e012b04074 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.421834:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762807.421839:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762807.421845:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b54a1e22/0x416193e012b04074 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.421858:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b54a1e22/0x416193e012b04074 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.424935:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b54a1e22/0x416193e012b04074 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.424945:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b54a1e22/0x416193e012b04074 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.424952:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b54a1e22/0x416193e012b04074 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.424958:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0e79000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762807.424975:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54a1e22/0x416193e012b04074 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.424983:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b54a1e22/0x416193e012b04074 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.424997:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b54a1e22/0x416193e012b04074 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.425003:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b54a1e22/0x416193e012b04074 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762807.425050:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015002112:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3286us (47040us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762807.425057:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 634 00000100:00100000:2.0:1789762807.425064:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330725576070 time 330725645948 state 0x2 ref 0 00000100:00100000:0.0:1789762807.427142:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.427189:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015002240 00000100:00100000:1.0:1789762807.427197:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015002240 00000100:00100000:1.0:1789762807.427233:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330725576070 time 330725645948 state 0x0 ref 1 00000100:00100000:1.0:1789762807.427250:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330775645948 time 330725645948 state 0x1 ref 1 00000100:00100000:1.0:1789762807.471835:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330775645948 time 330775729705 state 0x0 ref 1 00000100:00100000:1.0:1789762807.471856:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015002240:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.471890:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0407b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.471904:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.471907:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.471911:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0407b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.471922:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0407b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.475134:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0407b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.475144:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0407b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.475151:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0407b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.475157:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.475161:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0407b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.475167:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0407b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.475179:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0407b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.475183:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0407b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.475231:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015002240:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3382us (48090us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.475239:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 635 00000100:00100000:1.0:1789762807.475247:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330775645948 time 330775729705 state 0x2 ref 0 00000100:00100000:0.0:1789762807.477055:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.477118:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015002368 00000100:00100000:1.0:1789762807.477126:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015002368 00000100:00100000:1.0:1789762807.477159:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330775645948 time 330775729705 state 0x0 ref 1 00000100:00100000:1.0:1789762807.477169:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330825729705 time 330775729705 state 0x1 ref 1 00000100:00100000:1.0:1789762807.521896:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330825729705 time 330825792588 state 0x0 ref 1 00000100:00100000:1.0:1789762807.521924:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015002368:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.521967:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04082 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.521982:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.521985:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.521990:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04082 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.522001:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04082 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.525266:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04082 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.525278:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04082 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.525284:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04082 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.525290:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.525294:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04082 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.525301:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04082 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.525327:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04082 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.525333:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04082 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.525385:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015002368:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3472us (48331us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.525392:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 636 00000100:00100000:1.0:1789762807.525399:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330825729705 time 330825792588 state 0x2 ref 0 00000100:00100000:3.0:1789762807.527187:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.527218:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015002496 00000100:00100000:1.0:1789762807.527227:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015002496 00000100:00100000:1.0:1789762807.527269:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330825729705 time 330825792588 state 0x0 ref 1 00000100:00100000:1.0:1789762807.527280:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330875792588 time 330825792588 state 0x1 ref 1 00000100:00100000:2.0:1789762807.571973:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330875792588 time 330875870991 state 0x0 ref 1 00000100:00100000:2.0:1789762807.571994:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015002496:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762807.572027:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04089 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.572040:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762807.572044:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762807.572048:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04089 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.572071:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04089 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.575077:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04089 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.575103:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04089 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.575110:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04089 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.575115:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762807.575118:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04089 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.575124:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04089 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.575137:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04089 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.575142:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04089 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762807.575196:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015002496:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3209us (48010us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762807.575203:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 637 00000100:00100000:2.0:1789762807.575210:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330875792588 time 330875870991 state 0x2 ref 0 00000100:00100000:2.0:1789762807.576966:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.577018:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015002624 00000100:00100000:1.0:1789762807.577028:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015002624 00000100:00100000:1.0:1789762807.577070:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330875792588 time 330875870991 state 0x0 ref 1 00000100:00100000:1.0:1789762807.577115:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330925870991 time 330875870991 state 0x1 ref 1 00000100:00100000:1.0:1789762807.622041:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330925870991 time 330925936728 state 0x0 ref 1 00000100:00100000:1.0:1789762807.622066:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015002624:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.622133:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04090 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.622151:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.622155:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.622159:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04090 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.622170:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04090 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.624975:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04090 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.624990:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04090 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.625001:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04090 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.625010:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.625015:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04090 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.625025:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04090 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.625045:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04090 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.625054:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04090 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.625134:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015002624:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3075us (48165us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.625142:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 638 00000100:00100000:1.0:1789762807.625149:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330925870991 time 330925936728 state 0x2 ref 0 00000100:00100000:0.0:1789762807.627339:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.627375:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015002752 00000100:00100000:1.0:1789762807.627384:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015002752 00000100:00100000:1.0:1789762807.627420:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330925870991 time 330925936728 state 0x0 ref 1 00000100:00100000:1.0:1789762807.627431:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330975936728 time 330925936728 state 0x1 ref 1 00000100:00100000:1.0:1789762807.672119:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330975936728 time 330976014982 state 0x0 ref 1 00000100:00100000:1.0:1789762807.672140:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015002752:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.672178:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04097 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.672192:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.672197:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.672201:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04097 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.672212:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04097 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.675521:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04097 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.675532:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04097 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.675539:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04097 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.675544:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.675548:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04097 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.675553:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04097 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.675567:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04097 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.675571:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04097 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.675626:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015002752:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3490us (48287us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.675637:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 639 00000100:00100000:1.0:1789762807.675649:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330975936728 time 330976014982 state 0x2 ref 0 00000100:00100000:2.0:1789762807.677640:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.677677:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015002880 00000100:00100000:1.0:1789762807.677685:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015002880 00000100:00100000:1.0:1789762807.677727:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 330975936728 time 330976014982 state 0x0 ref 1 00000100:00100000:1.0:1789762807.677742:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331026014982 time 330976014982 state 0x1 ref 1 00000100:00100000:2.0:1789762807.722195:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331026014982 time 331026091520 state 0x0 ref 1 00000100:00100000:2.0:1789762807.722216:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015002880:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762807.722257:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b0409e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.722271:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762807.722275:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762807.722279:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b0409e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.722292:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b0409e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.725569:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b0409e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.725594:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b0409e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.725601:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b0409e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.725607:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81682400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762807.725610:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b0409e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.725618:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b0409e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.725632:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b0409e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.725638:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b0409e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762807.725684:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015002880:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3474us (48045us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762807.725691:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 640 00000100:00100000:2.0:1789762807.725698:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331026014982 time 331026091520 state 0x2 ref 0 00000100:00100000:2.0:1789762807.727874:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.727929:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015003008 00000100:00100000:1.0:1789762807.727937:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015003008 00000100:00100000:1.0:1789762807.727980:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331026014982 time 331026091520 state 0x0 ref 1 00000100:00100000:1.0:1789762807.727990:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331076091520 time 331026091520 state 0x1 ref 1 00010000:00010000:0.0:1789762807.735156:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762807.772243:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331076091520 time 331076141204 state 0x0 ref 1 00000100:00100000:1.0:1789762807.772263:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015003008:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.772294:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040a5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.772320:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.772324:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.772329:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040a5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.772339:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040a5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.775185:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040a5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.775195:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040a5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.775201:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040a5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.775206:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.775210:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040a5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.775216:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040a5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.775228:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040a5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.775233:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040a5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.775285:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015003008:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3028us (47412us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.775292:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 641 00000100:00100000:1.0:1789762807.775312:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331076091520 time 331076141204 state 0x2 ref 0 00000100:00100000:2.0:1789762807.777455:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.777489:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015003136 00000100:00100000:1.0:1789762807.777497:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015003136 00000100:00100000:1.0:1789762807.777531:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331076091520 time 331076141204 state 0x0 ref 1 00000100:00100000:1.0:1789762807.777541:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331126141204 time 331076141204 state 0x1 ref 1 00000100:00100000:1.0:1789762807.822293:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331126141204 time 331126189876 state 0x0 ref 1 00000100:00100000:1.0:1789762807.822326:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015003136:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.822361:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ac lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.822375:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.822378:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.822383:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ac lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.822393:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ac lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.825292:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ac lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.825312:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ac lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.825319:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ac lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.825325:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.825328:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ac lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.825335:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ac lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.825349:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ac lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.825355:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ac lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.825414:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015003136:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3094us (47958us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.825425:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 642 00000100:00100000:1.0:1789762807.825433:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331126141204 time 331126189876 state 0x2 ref 0 00000100:00100000:3.0:1789762807.827152:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.827182:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015003264 00000100:00100000:1.0:1789762807.827190:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015003264 00000100:00100000:1.0:1789762807.827222:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331126141204 time 331126189876 state 0x0 ref 1 00000100:00100000:1.0:1789762807.827233:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331176189876 time 331126189876 state 0x1 ref 1 00000100:00100000:2.0:1789762807.872388:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331176189876 time 331176286997 state 0x0 ref 1 00000100:00100000:2.0:1789762807.872408:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015003264:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762807.872448:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040b3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.872461:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762807.872464:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762807.872468:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040b3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.872479:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040b3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.875231:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040b3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.875242:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040b3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.875249:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040b3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.875254:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81682400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762807.875258:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040b3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.875271:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040b3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.875284:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040b3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762807.875289:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040b3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762807.875344:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015003264:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2941us (48193us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762807.875351:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 643 00000100:00100000:2.0:1789762807.875359:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331176189876 time 331176286997 state 0x2 ref 0 00000100:00100000:0.0:1789762807.877205:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.877254:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015003392 00000100:00100000:1.0:1789762807.877262:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015003392 00000100:00100000:1.0:1789762807.877296:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331176189876 time 331176286997 state 0x0 ref 1 00000100:00100000:1.0:1789762807.877323:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331226286997 time 331176286997 state 0x1 ref 1 00000100:00100000:3.0:1789762807.906224:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762807.906315:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762807.906331:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015003520 00000100:00100000:0.0:1789762807.906342:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015003520 00000100:00100000:3.0:1789762807.906343:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762807.906366:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015003648 00000100:00100000:2.0:1789762807.906374:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015003648 00000100:00100000:3.0:1789762807.906378:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.906400:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015003776 00000100:00100000:0.0:1789762807.906402:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 302 00000100:00100000:1.0:1789762807.906408:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015003776 00000100:00100000:3.0:1789762807.906411:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015003904 00000100:00100000:3.0:1789762807.906417:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015003904 00000100:00100000:0.0:1789762807.906418:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015003520:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762807.906440:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 243 00000100:00100000:2.0:1789762807.906451:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2364:x1876702015003648:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762807.906460:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 121 00000100:00100000:3.0:1789762807.906464:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 244 00000100:00100000:1.0:1789762807.906472:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015003776:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762807.906479:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015003904:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762807.906485:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015003520:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 70us (269us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762807.906493:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 302 00000100:00100000:2.0:1789762807.906497:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2364:x1876702015003648:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 48us (193us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762807.906505:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 243 00000100:00100000:1.0:1789762807.906515:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015003776:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 46us (170us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762807.906522:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015003904:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 47us (143us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762807.906526:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 121 00000100:00080000:2.0:1789762807.906531:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762807.906535:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 244 00000100:00080000:2.0:1789762807.906538:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762807.922469:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331226286997 time 331226364235 state 0x0 ref 1 00000100:00100000:1.0:1789762807.922496:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015003392:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.922532:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ba lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.922547:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.922550:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.922555:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ba lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.922565:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ba lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.925850:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ba lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.925861:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ba lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.925868:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ba lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.925874:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.925878:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ba lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.925885:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ba lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.925898:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ba lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.925908:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040ba lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.925955:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015003392:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3470us (48751us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.925962:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 644 00000100:00100000:1.0:1789762807.925970:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331226286997 time 331226364235 state 0x2 ref 0 00000100:00100000:3.0:1789762807.927231:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc555500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022166656:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762807.927272:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762807.927316:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022166656 00000100:00100000:0.0:1789762807.927324:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022166656 00000100:00100000:0.0:1789762807.927370:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 103 00000100:00100000:0.0:1789762807.927377:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022166656:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762807.927400:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3746562048 avail 3744464896 00000100:00100000:0.0:1789762807.927438:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022166656:12345-0@lo:13:osp-pre-0-0.0 Request processed in 62us (166us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762807.927444:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 103 00000100:00100000:3.0:1789762807.927489:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc555500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022166656:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762807.927922:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.927960:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015004032 00000100:00100000:1.0:1789762807.927965:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015004032 00000100:00100000:1.0:1789762807.927993:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331226286997 time 331226364235 state 0x0 ref 1 00000100:00100000:1.0:1789762807.928003:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331276364235 time 331226364235 state 0x1 ref 1 00000100:00100000:1.0:1789762807.972547:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331276364235 time 331276439919 state 0x0 ref 1 00000100:00100000:1.0:1789762807.972576:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015004032:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762807.972618:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040c1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.972635:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.972639:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762807.972646:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040c1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.972667:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040c1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.976294:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040c1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.976323:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040c1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.976334:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040c1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.976343:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762807.976348:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040c1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.976358:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040c1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.976372:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040c1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762807.976378:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040c1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762807.976437:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015004032:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3872us (48513us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762807.976446:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 645 00000100:00100000:1.0:1789762807.976455:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331276364235 time 331276439919 state 0x2 ref 0 00000100:00100000:2.0:1789762807.978172:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762807.978208:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015004160 00000100:00100000:1.0:1789762807.978216:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015004160 00000100:00100000:1.0:1789762807.978257:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331276364235 time 331276439919 state 0x0 ref 1 00000100:00100000:1.0:1789762807.978272:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331326439919 time 331276439919 state 0x1 ref 1 00000100:00100000:2.0:1789762808.022633:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331326439919 time 331326530763 state 0x0 ref 1 00000100:00100000:2.0:1789762808.022666:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015004160:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762808.022700:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040c8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.022713:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762808.022717:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762808.022721:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040c8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.022731:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040c8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.025550:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040c8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.025562:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040c8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.025570:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040c8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.025575:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81682400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762808.025579:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040c8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.025596:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040c8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.025610:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040c8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.025615:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b040c8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762808.025661:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015004160:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3001us (47491us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762808.025669:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 646 00000100:00100000:0.0:1789762808.025676:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331326439919 time 331326530763 state 0x2 ref 0 00000100:00100000:0.0:1789762808.027514:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.027560:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015004288 00000100:00100000:1.0:1789762808.027569:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015004288 00000100:00100000:1.0:1789762808.027603:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331326439919 time 331326530763 state 0x0 ref 1 00000100:00100000:1.0:1789762808.027614:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331376530763 time 331326530763 state 0x1 ref 1 00000100:00100000:1.0:1789762808.072692:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331376530763 time 331376589316 state 0x0 ref 1 00000100:00100000:1.0:1789762808.072714:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015004288:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.072749:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040cf lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.072764:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.072768:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.072773:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040cf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.072783:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040cf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.075476:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040cf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.075490:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040cf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.075501:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040cf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.075508:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.075512:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040cf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.075519:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040cf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.075532:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040cf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.075537:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040cf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.075588:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015004288:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2880us (48073us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.075594:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 647 00000100:00100000:1.0:1789762808.075602:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331376530763 time 331376589316 state 0x2 ref 0 00000100:00100000:3.0:1789762808.078001:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.078047:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015004416 00000100:00100000:1.0:1789762808.078054:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015004416 00000100:00100000:1.0:1789762808.078108:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331376530763 time 331376589316 state 0x0 ref 1 00000100:00100000:1.0:1789762808.078119:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331426589316 time 331376589316 state 0x1 ref 1 00000100:00100000:1.0:1789762808.122740:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331426589316 time 331426637680 state 0x0 ref 1 00000100:00100000:1.0:1789762808.122766:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015004416:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.122799:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040d6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.122813:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.122818:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.122824:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040d6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.122839:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040d6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.125855:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040d6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.125866:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040d6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.125873:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040d6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.125879:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.125883:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040d6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.125890:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040d6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.125906:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040d6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.125914:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040d6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.125974:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015004416:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3219us (47972us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.125982:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 648 00000100:00100000:1.0:1789762808.125991:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331426589316 time 331426637680 state 0x2 ref 0 00000100:00100000:3.0:1789762808.128472:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.128504:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015004544 00000100:00100000:1.0:1789762808.128512:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015004544 00000100:00100000:1.0:1789762808.128546:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331426589316 time 331426637680 state 0x0 ref 1 00000100:00100000:1.0:1789762808.128556:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331476637680 time 331426637680 state 0x1 ref 1 00000100:00100000:0.0:1789762808.172813:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331476637680 time 331476712167 state 0x0 ref 1 00000100:00100000:0.0:1789762808.172838:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015004544:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762808.172878:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b040dd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.172892:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762808.172896:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762808.172900:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b040dd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.172925:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b040dd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.175847:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b040dd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.175856:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b040dd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.175862:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b040dd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.175867:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762808.175870:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b040dd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.175875:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b040dd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.175894:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b040dd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.175899:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b040dd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762808.175945:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015004544:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3113us (47473us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762808.175955:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 649 00000100:00100000:2.0:1789762808.175964:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331476637680 time 331476712167 state 0x2 ref 0 00000100:00100000:3.0:1789762808.178153:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.178233:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015004672 00000100:00100000:1.0:1789762808.178240:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015004672 00000100:00100000:1.0:1789762808.178275:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331476637680 time 331476712167 state 0x0 ref 1 00000100:00100000:1.0:1789762808.178285:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331526712167 time 331476712167 state 0x1 ref 1 00000100:00100000:1.0:1789762808.222885:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331526712167 time 331526782450 state 0x0 ref 1 00000100:00100000:1.0:1789762808.222907:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015004672:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.222947:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040e4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.222964:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.222969:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.222976:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040e4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.222989:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040e4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.225700:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040e4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.225711:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040e4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.225730:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040e4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.225736:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.225740:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040e4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.225746:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040e4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.225760:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040e4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.225766:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040e4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.225816:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015004672:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2915us (47663us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.225823:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 650 00000100:00100000:1.0:1789762808.225832:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331526712167 time 331526782450 state 0x2 ref 0 00000100:00100000:3.0:1789762808.228081:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.228134:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015004800 00000100:00100000:1.0:1789762808.228142:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015004800 00000100:00100000:1.0:1789762808.228175:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331526712167 time 331526782450 state 0x0 ref 1 00000100:00100000:1.0:1789762808.228185:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331576782450 time 331526782450 state 0x1 ref 1 00000100:00100000:1.0:1789762808.272936:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331576782450 time 331576831157 state 0x0 ref 1 00000100:00100000:1.0:1789762808.272960:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015004800:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.272997:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040eb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.273011:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.273015:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.273019:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040eb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.273030:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040eb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.275930:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040eb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.275942:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040eb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.275949:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040eb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.275955:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.275958:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040eb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.275964:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040eb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.275977:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040eb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.275982:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040eb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.276033:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015004800:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3082us (47953us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.276041:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 651 00000100:00100000:1.0:1789762808.276049:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331576782450 time 331576831157 state 0x2 ref 0 00000100:00100000:3.0:1789762808.278144:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.278181:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015004928 00000100:00100000:1.0:1789762808.278188:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015004928 00000100:00100000:1.0:1789762808.278224:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331576782450 time 331576831157 state 0x0 ref 1 00000100:00100000:1.0:1789762808.278236:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331626831157 time 331576831157 state 0x1 ref 1 00000100:00100000:2.0:1789762808.323015:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331626831157 time 331626912017 state 0x0 ref 1 00000100:00100000:2.0:1789762808.323036:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015004928:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762808.323071:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d49054cb/0x416193e012b040f2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.323104:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762808.323108:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762808.323113:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d49054cb/0x416193e012b040f2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.323125:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d49054cb/0x416193e012b040f2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.326398:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d49054cb/0x416193e012b040f2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.326409:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d49054cb/0x416193e012b040f2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.326416:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d49054cb/0x416193e012b040f2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.326422:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc36a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762808.326426:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d49054cb/0x416193e012b040f2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.326432:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d49054cb/0x416193e012b040f2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.326449:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d49054cb/0x416193e012b040f2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.326458:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d49054cb/0x416193e012b040f2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762808.326520:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad83100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015004928:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3488us (48376us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762808.326530:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 652 00000100:00100000:2.0:1789762808.326549:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331626831157 time 331626912017 state 0x2 ref 0 00000100:00100000:0.0:1789762808.328620:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762808.328656:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015005056 00000100:00100000:2.0:1789762808.328664:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015005056 00000100:00100000:2.0:1789762808.328698:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331626831157 time 331626912017 state 0x0 ref 1 00000100:00100000:2.0:1789762808.328708:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331676912017 time 331626912017 state 0x1 ref 1 00000100:00100000:1.0:1789762808.373105:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331676912017 time 331676983984 state 0x0 ref 1 00000100:00100000:1.0:1789762808.373123:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015005056:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.373153:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040f9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.373166:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.373170:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.373174:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040f9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.373184:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040f9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.376159:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040f9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.376169:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040f9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.376176:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040f9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.376185:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.376189:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040f9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.376195:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040f9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.376207:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040f9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.376211:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b040f9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.376259:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015005056:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3140us (47641us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.376266:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 653 00000100:00100000:1.0:1789762808.376274:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331676912017 time 331676983984 state 0x2 ref 0 00000100:00100000:0.0:1789762808.378244:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.378275:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015005184 00000100:00100000:1.0:1789762808.378283:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015005184 00000100:00100000:1.0:1789762808.378332:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331676912017 time 331676983984 state 0x0 ref 1 00000100:00100000:1.0:1789762808.378342:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331726983984 time 331676983984 state 0x1 ref 1 00000100:00100000:1.0:1789762808.423139:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331726983984 time 331727037562 state 0x0 ref 1 00000100:00100000:1.0:1789762808.423158:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015005184:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.423191:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04100 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.423205:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.423209:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.423213:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04100 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.423224:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04100 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.426217:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04100 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.426228:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04100 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.426235:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04100 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.426241:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.426244:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04100 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.426250:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04100 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.426265:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04100 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.426271:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04100 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.426334:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015005184:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3181us (48099us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.426341:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 654 00000100:00100000:1.0:1789762808.426348:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331726983984 time 331727037562 state 0x2 ref 0 00000100:00100000:0.0:1789762808.428344:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.428377:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015005312 00000100:00100000:1.0:1789762808.428384:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015005312 00000100:00100000:1.0:1789762808.428415:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331726983984 time 331727037562 state 0x0 ref 1 00000100:00100000:1.0:1789762808.428425:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331777037562 time 331727037562 state 0x1 ref 1 00000100:00100000:0.0:1789762808.439238:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022166784:0@lo:400:kworker.0 00000100:00100000:1.0:1789762808.439249:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022167040:0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.439265:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc556300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022166912:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762808.439290:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762808.439310:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762808.439346:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762808.439349:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022166784 00000100:00100000:2.0:1789762808.439355:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022166784 00000100:00100000:0.0:1789762808.439365:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022167296:0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.439379:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022167424:0@lo:400:kworker.0 00000100:00100000:1.0:1789762808.439385:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022167040 00000100:00100000:1.0:1789762808.439390:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022167040 00000100:00100000:0.0:1789762808.439400:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762808.439400:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 303 00000100:00100000:3.0:1789762808.439417:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762808.439418:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85086c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022166784:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762808.439430:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022167296 00000100:00100000:0.0:1789762808.439433:0:3272:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789762808.439434:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022167296 00000100:00100000:0.0:1789762808.439442:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022167168:0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.439469:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022167424 00000100:00100000:2.0:1789762808.439470:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85086c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022166784:12345-0@lo:400:kworker.0 Request processed in 58us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762808.439475:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022167424 00000100:00100000:1.0:1789762808.439477:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:0.0:1789762808.439482:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762808.439486:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 303 00000100:00100000:1.0:1789762808.439492:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc557480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022167040:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762808.439523:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022166784:0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.439524:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00000100:00080000:2.0:1789762808.439526:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762808.439534:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762808.439540:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022167424:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762808.439547:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc557480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022167040:12345-0@lo:400:kworker.0 Request processed in 58us (250us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762808.439554:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:2.0:1789762808.439559:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022167168 00000100:00100000:0.0:1789762808.439562:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022166912 00000100:00100000:2.0:1789762808.439564:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022167168 00000100:00100000:0.0:1789762808.439567:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022166912 00000100:00100000:1.0:1789762808.439575:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:1.0:1789762808.439586:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022167296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762808.439593:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad83800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3273:x1876702022167424:12345-0@lo:400:kworker.0 Request processed in 56us (179us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762808.439601:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000100:00100000:2.0:1789762808.439607:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 247 00000100:00100000:0.0:1789762808.439611:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 104 00000100:00100000:2.0:1789762808.439619:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022167168:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762808.439624:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022166912:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762808.439631:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022167296:12345-0@lo:400:kworker.0 Request processed in 47us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762808.439637:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00100000:3.0:1789762808.439643:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022167424:0@lo:400:kworker.0 00002000:00000024:0.0:1789762808.439651:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3851419648 avail 3849322496 00000100:00100000:2.0:1789762808.439669:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022167168:12345-0@lo:400:kworker.0 Request processed in 53us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762808.439676:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 247 00000100:00100000:1.0:1789762808.439678:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022167040:0@lo:400:kworker.0 00000100:00080000:3.0:1789762808.439680:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762808.439700:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022166912:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (355us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762808.439707:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 104 00000100:00100000:3.0:1789762808.439715:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc556300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022166912:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762808.439744:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022167168:0@lo:400:kworker.0 00000100:00100000:0.0:1789762808.439762:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022167296:0@lo:400:kworker.0 00000100:00100000:2.0:1789762808.473383:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331777037562 time 331777265033 state 0x0 ref 1 00000100:00100000:2.0:1789762808.473447:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015005312:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762808.473524:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fed4c/0x416193e012b04107 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.473547:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762808.473553:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762808.473561:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fed4c/0x416193e012b04107 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.473579:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d82fed4c/0x416193e012b04107 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.476869:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fed4c/0x416193e012b04107 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.476879:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fed4c/0x416193e012b04107 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.476884:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d82fed4c/0x416193e012b04107 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.476889:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec04a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762808.476893:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fed4c/0x416193e012b04107 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.476902:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fed4c/0x416193e012b04107 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.476934:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d82fed4c/0x416193e012b04107 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.476943:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d82fed4c/0x416193e012b04107 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762808.476994:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015005312:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3567us (48652us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762808.477001:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 655 00000100:00100000:2.0:1789762808.477011:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331777037562 time 331777265033 state 0x2 ref 0 00000100:00100000:2.0:1789762808.479179:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762808.479210:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015005440 00000100:00100000:3.0:1789762808.479218:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015005440 00000100:00100000:3.0:1789762808.479256:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331777037562 time 331777265033 state 0x0 ref 1 00000100:00100000:3.0:1789762808.479267:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331827265033 time 331777265033 state 0x1 ref 1 00000100:00100000:1.0:1789762808.523459:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331827265033 time 331827352475 state 0x0 ref 1 00000100:00100000:1.0:1789762808.523485:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015005440:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.523526:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0410e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.523543:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.523548:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.523554:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0410e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.523568:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0410e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.526901:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0410e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.526919:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0410e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.526929:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0410e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.526939:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.526944:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0410e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.526955:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0410e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.526973:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0410e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.526981:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0410e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.527047:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015005440:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3568us (47868us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.527057:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 656 00000100:00100000:1.0:1789762808.527067:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331827265033 time 331827352475 state 0x2 ref 0 00000100:00100000:3.0:1789762808.529368:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.529407:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015005568 00000100:00100000:1.0:1789762808.529416:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015005568 00000100:00100000:1.0:1789762808.529458:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331827265033 time 331827352475 state 0x0 ref 1 00000100:00100000:1.0:1789762808.529472:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331877352475 time 331827352475 state 0x1 ref 1 00000100:00100000:1.0:1789762808.573514:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331877352475 time 331877410386 state 0x0 ref 1 00000100:00100000:1.0:1789762808.573535:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015005568:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.573568:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04115 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.573583:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.573588:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.573595:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04115 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.573609:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04115 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.576411:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04115 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.576422:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04115 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.576432:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04115 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.576442:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.576446:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04115 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.576455:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04115 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.576470:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04115 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.576479:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04115 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.576533:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015005568:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3003us (47166us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.576543:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 657 00000100:00100000:1.0:1789762808.576553:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331877352475 time 331877410386 state 0x2 ref 0 00000100:00100000:0.0:1789762808.578931:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.578974:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015005696 00000100:00100000:1.0:1789762808.578982:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015005696 00000100:00100000:1.0:1789762808.579016:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331877352475 time 331877410386 state 0x0 ref 1 00000100:00100000:1.0:1789762808.579029:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331927410386 time 331877410386 state 0x1 ref 1 00000100:00100000:2.0:1789762808.623608:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331927410386 time 331927503592 state 0x0 ref 1 00000100:00100000:2.0:1789762808.623629:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015005696:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762808.623665:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000987b5c94/0x416193e012b0411c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.623679:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762808.623683:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762808.623688:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000987b5c94/0x416193e012b0411c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.623698:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000987b5c94/0x416193e012b0411c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.626914:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000987b5c94/0x416193e012b0411c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.626925:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000987b5c94/0x416193e012b0411c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.626932:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000987b5c94/0x416193e012b0411c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.626938:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e879db200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762808.626942:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000987b5c94/0x416193e012b0411c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.626948:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000987b5c94/0x416193e012b0411c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.626962:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000987b5c94/0x416193e012b0411c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.626967:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000987b5c94/0x416193e012b0411c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762808.627014:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015005696:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3391us (48090us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762808.627033:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 658 00000100:00100000:0.0:1789762808.627041:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331927410386 time 331927503592 state 0x2 ref 0 00000100:00100000:0.0:1789762808.629005:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762808.629046:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015005824 00000100:00100000:3.0:1789762808.629054:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015005824 00000100:00100000:3.0:1789762808.629106:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331927410386 time 331927503592 state 0x0 ref 1 00000100:00100000:3.0:1789762808.629118:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331977503592 time 331927503592 state 0x1 ref 1 00000100:00100000:1.0:1789762808.673726:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331977503592 time 331977619576 state 0x0 ref 1 00000100:00100000:1.0:1789762808.673752:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015005824:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.673791:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04123 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.673816:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.673821:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.673829:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04123 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.673844:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04123 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.676691:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04123 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.676704:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04123 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.676714:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04123 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.676722:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.676727:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04123 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.676737:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04123 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.676755:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04123 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.676763:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04123 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.676822:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015005824:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3077us (47815us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.676832:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 659 00000100:00100000:1.0:1789762808.676842:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331977503592 time 331977619576 state 0x2 ref 0 00000100:00100000:3.0:1789762808.678997:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.679046:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015005952 00000100:00100000:1.0:1789762808.679056:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015005952 00000100:00100000:1.0:1789762808.679119:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 331977503592 time 331977619576 state 0x0 ref 1 00000100:00100000:1.0:1789762808.679135:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332027619576 time 331977619576 state 0x1 ref 1 00000100:00100000:1.0:1789762808.723805:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332027619576 time 332027699219 state 0x0 ref 1 00000100:00100000:1.0:1789762808.723827:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015005952:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.723865:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0412a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.723885:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762808.723898:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.723905:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0412a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.723922:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0412a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.726943:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0412a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.726952:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0412a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.726959:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0412a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.726965:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.726968:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0412a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.726975:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0412a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.726989:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0412a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.726994:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0412a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.727041:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015005952:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3221us (48045us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.727047:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 660 00000100:00100000:1.0:1789762808.727054:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332027619576 time 332027699219 state 0x2 ref 0 00000100:00100000:3.0:1789762808.729172:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.729204:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015006080 00000100:00100000:1.0:1789762808.729212:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015006080 00000100:00100000:1.0:1789762808.729253:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332027619576 time 332027699219 state 0x0 ref 1 00000100:00100000:1.0:1789762808.729267:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332077699219 time 332027699219 state 0x1 ref 1 00010000:00010000:3.0:1789762808.759220:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762808.773887:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332077699219 time 332077784909 state 0x0 ref 1 00000100:00100000:0.0:1789762808.773907:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015006080:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762808.773947:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a83c44a7/0x416193e012b04131 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.773961:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762808.773964:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762808.773969:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a83c44a7/0x416193e012b04131 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762808.773980:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a83c44a7/0x416193e012b04131 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.777042:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a83c44a7/0x416193e012b04131 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.777056:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a83c44a7/0x416193e012b04131 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.777067:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a83c44a7/0x416193e012b04131 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.777075:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063a400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762808.777080:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a83c44a7/0x416193e012b04131 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.777118:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a83c44a7/0x416193e012b04131 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.777136:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a83c44a7/0x416193e012b04131 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.777144:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a83c44a7/0x416193e012b04131 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762808.777194:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015006080:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3292us (48024us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762808.777200:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 661 00000100:00100000:2.0:1789762808.777215:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332077699219 time 332077784909 state 0x2 ref 0 00000100:00100000:3.0:1789762808.779083:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762808.779133:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015006208 00000100:00100000:2.0:1789762808.779141:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015006208 00000100:00100000:2.0:1789762808.779175:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332077699219 time 332077784909 state 0x0 ref 1 00000100:00100000:2.0:1789762808.779186:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332127784909 time 332077784909 state 0x1 ref 1 00000100:00100000:1.0:1789762808.823988:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332127784909 time 332127884307 state 0x0 ref 1 00000100:00100000:1.0:1789762808.824010:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015006208:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.824045:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04138 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.824058:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.824061:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.824066:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04138 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.824077:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04138 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.827392:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04138 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.827408:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04138 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.827417:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04138 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.827425:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.827430:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04138 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.827440:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04138 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.827459:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04138 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.827468:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04138 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.827520:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad83480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015006208:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3516us (48437us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.827528:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 662 00000100:00100000:1.0:1789762808.827538:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332127784909 time 332127884307 state 0x2 ref 0 00000100:00100000:0.0:1789762808.829956:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.829998:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015006336 00000100:00100000:1.0:1789762808.830006:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015006336 00000100:00100000:1.0:1789762808.830044:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332127784909 time 332127884307 state 0x0 ref 1 00000100:00100000:1.0:1789762808.830055:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332177884307 time 332127884307 state 0x1 ref 1 00000100:00100000:1.0:1789762808.874034:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332177884307 time 332177932233 state 0x0 ref 1 00000100:00100000:1.0:1789762808.874053:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015006336:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.874104:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0413f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.874118:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.874121:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.874126:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0413f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.874136:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0413f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.876816:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0413f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.876830:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0413f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.876841:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0413f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.876850:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.876854:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0413f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.876865:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0413f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.876881:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0413f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.876890:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0413f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.876941:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015006336:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2892us (46988us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.876951:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 663 00000100:00100000:1.0:1789762808.876961:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332177884307 time 332177932233 state 0x2 ref 0 00000100:00100000:0.0:1789762808.879181:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.879212:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015006464 00000100:00100000:1.0:1789762808.879220:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015006464 00000100:00100000:1.0:1789762808.879254:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332177884307 time 332177932233 state 0x0 ref 1 00000100:00100000:1.0:1789762808.879268:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332227932233 time 332177932233 state 0x1 ref 1 00000100:00100000:2.0:1789762808.924162:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332227932233 time 332228053462 state 0x0 ref 1 00000100:00100000:2.0:1789762808.924194:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015006464:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762808.924253:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b04146 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.924272:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762808.924278:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762808.924286:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b04146 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.924318:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b04146 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.927913:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b04146 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.927930:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b04146 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.927955:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b04146 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.927964:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762808.927969:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b04146 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.927980:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b04146 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.928001:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b04146 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762808.928010:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b04146 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762808.928079:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015006464:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3895us (48897us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762808.928130:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 664 00000100:00100000:2.0:1789762808.928142:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332227932233 time 332228053462 state 0x2 ref 0 00000100:00100000:3.0:1789762808.930387:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762808.930431:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015006592 00000100:00100000:2.0:1789762808.930442:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015006592 00000100:00100000:2.0:1789762808.930490:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332227932233 time 332228053462 state 0x0 ref 1 00000100:00100000:2.0:1789762808.930520:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332278053462 time 332228053462 state 0x1 ref 1 00000100:00100000:1.0:1789762808.974263:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332278053462 time 332278160200 state 0x0 ref 1 00000100:00100000:1.0:1789762808.974283:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015006592:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762808.974339:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0414d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.974352:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.974356:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762808.974366:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0414d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.974376:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0414d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.977414:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0414d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.977425:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0414d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.977432:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0414d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.977441:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762808.977445:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0414d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.977451:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0414d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.977463:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0414d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762808.977468:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b0414d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762808.977528:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad82300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015006592:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3250us (47142us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762808.977536:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 665 00000100:00100000:1.0:1789762808.977544:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332278053462 time 332278160200 state 0x2 ref 0 00000100:00100000:3.0:1789762808.979458:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762808.979491:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015006720 00000100:00100000:1.0:1789762808.979498:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015006720 00000100:00100000:1.0:1789762808.979533:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332278053462 time 332278160200 state 0x0 ref 1 00000100:00100000:1.0:1789762808.979542:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332328160200 time 332278160200 state 0x1 ref 1 00000100:00100000:1.0:1789762809.024332:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332328160200 time 332328227965 state 0x0 ref 1 00000100:00100000:1.0:1789762809.024367:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015006720:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:3.0:1789762809.024394:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762809.024408:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04154 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.024427:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.024432:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.024438:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04154 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.024449:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04154 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.027359:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04154 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.027371:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04154 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.027378:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04154 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.027384:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762809.027389:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04154 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.027395:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04154 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.027410:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04154 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.027416:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04154 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762809.027466:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015006720:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3119us (48008us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762809.027483:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 666 00000100:00100000:2.0:1789762809.027491:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332328160200 time 332328227965 state 0x2 ref 0 00000100:00100000:0.0:1789762809.029895:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.029938:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015006848 00000100:00100000:1.0:1789762809.029946:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015006848 00000100:00100000:1.0:1789762809.029983:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332328160200 time 332328227965 state 0x0 ref 1 00000100:00100000:1.0:1789762809.029993:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332378227965 time 332328227965 state 0x1 ref 1 00000100:00100000:2.0:1789762809.074440:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332378227965 time 332378335036 state 0x0 ref 1 00000100:00100000:2.0:1789762809.074461:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015006848:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762809.074495:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0415b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.074509:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.074513:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.074517:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0415b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.074529:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0415b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.077890:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0415b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.077902:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0415b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.077922:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0415b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.077928:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762809.077932:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0415b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.077939:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0415b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.077954:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0415b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.077959:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a80a8279/0x416193e012b0415b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762809.078010:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015006848:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3554us (48114us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762809.078017:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 667 00000100:00100000:0.0:1789762809.078025:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332378227965 time 332378335036 state 0x2 ref 0 00000100:00100000:0.0:1789762809.080083:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762809.080151:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015006976 00000100:00100000:2.0:1789762809.080161:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015006976 00000100:00100000:2.0:1789762809.080200:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332378227965 time 332378335036 state 0x0 ref 1 00000100:00100000:2.0:1789762809.080211:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332428335036 time 332378335036 state 0x1 ref 1 00000100:00100000:1.0:1789762809.124532:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332428335036 time 332428423825 state 0x0 ref 1 00000100:00100000:1.0:1789762809.124558:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015006976:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762809.124599:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04162 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.124624:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.124629:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.124636:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04162 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.124651:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04162 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.127564:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04162 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.127576:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04162 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.127583:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04162 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.127589:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762809.127592:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04162 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.127599:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04162 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.127613:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04162 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.127623:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04162 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762809.127678:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015006976:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3127us (47597us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762809.127685:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 668 00000100:00100000:1.0:1789762809.127693:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332428335036 time 332428423825 state 0x2 ref 0 00000100:00100000:3.0:1789762809.129656:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.129689:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015007104 00000100:00100000:1.0:1789762809.129697:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015007104 00000100:00100000:1.0:1789762809.129745:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332428335036 time 332428423825 state 0x0 ref 1 00000100:00100000:1.0:1789762809.129760:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332478423825 time 332428423825 state 0x1 ref 1 00000100:00100000:2.0:1789762809.174622:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332478423825 time 332478519275 state 0x0 ref 1 00000100:00100000:2.0:1789762809.174655:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015007104:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762809.174697:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04169 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.174715:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.174720:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.174727:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04169 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.174742:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04169 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.177651:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04169 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.177662:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04169 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.177669:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04169 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.177674:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762809.177678:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04169 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.177684:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04169 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.177705:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04169 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.177715:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04169 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762809.177773:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015007104:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3122us (48117us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762809.177780:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 669 00000100:00100000:2.0:1789762809.177789:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332478423825 time 332478519275 state 0x2 ref 0 00000100:00100000:2.0:1789762809.179660:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.179725:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015007232 00000100:00100000:1.0:1789762809.179735:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015007232 00000100:00100000:1.0:1789762809.179778:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332478423825 time 332478519275 state 0x0 ref 1 00000100:00100000:1.0:1789762809.179793:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332528519275 time 332478519275 state 0x1 ref 1 00000100:00100000:0.0:1789762809.224701:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332528519275 time 332528594369 state 0x0 ref 1 00000100:00100000:0.0:1789762809.224726:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015007232:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762809.224780:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006889adc7/0x416193e012b04170 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.224797:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762809.224801:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762809.224807:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006889adc7/0x416193e012b04170 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.224818:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006889adc7/0x416193e012b04170 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.227853:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006889adc7/0x416193e012b04170 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.227863:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006889adc7/0x416193e012b04170 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.227870:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006889adc7/0x416193e012b04170 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.227875:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81682600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762809.227879:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006889adc7/0x416193e012b04170 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.227886:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006889adc7/0x416193e012b04170 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.227911:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006889adc7/0x416193e012b04170 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.227917:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006889adc7/0x416193e012b04170 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762809.227970:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015007232:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3250us (48310us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762809.227980:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 670 00000100:00100000:0.0:1789762809.227990:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332528519275 time 332528594369 state 0x2 ref 0 00000100:00100000:0.0:1789762809.230127:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762809.230182:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015007360 00000100:00100000:2.0:1789762809.230192:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015007360 00000100:00100000:2.0:1789762809.230235:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332528519275 time 332528594369 state 0x0 ref 1 00000100:00100000:2.0:1789762809.230250:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332578594369 time 332528594369 state 0x1 ref 1 00000100:00100000:1.0:1789762809.274834:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332578594369 time 332578728558 state 0x0 ref 1 00000100:00100000:1.0:1789762809.274856:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015007360:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762809.274904:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04177 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.274919:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.274923:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.274928:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04177 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.274943:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04177 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.278145:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04177 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.278157:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04177 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.278163:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04177 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.278168:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762809.278171:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04177 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.278178:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04177 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.278196:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04177 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.278202:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04177 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762809.278252:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015007360:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3402us (48124us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762809.278259:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 671 00000100:00100000:1.0:1789762809.278267:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332578594369 time 332578728558 state 0x2 ref 0 00000100:00100000:3.0:1789762809.280224:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.280276:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015007488 00000100:00100000:1.0:1789762809.280285:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015007488 00000100:00100000:1.0:1789762809.280341:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332578594369 time 332578728558 state 0x0 ref 1 00000100:00100000:1.0:1789762809.280353:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332628728558 time 332578728558 state 0x1 ref 1 00000100:00100000:2.0:1789762809.324941:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332628728558 time 332628836342 state 0x0 ref 1 00000100:00100000:2.0:1789762809.324964:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015007488:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762809.325011:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d711201f/0x416193e012b0417e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.325040:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.325044:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.325049:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d711201f/0x416193e012b0417e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.325060:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d711201f/0x416193e012b0417e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.328023:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d711201f/0x416193e012b0417e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.328035:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d711201f/0x416193e012b0417e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.328044:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d711201f/0x416193e012b0417e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.328051:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c6f600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762809.328056:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d711201f/0x416193e012b0417e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.328066:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d711201f/0x416193e012b0417e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.328103:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d711201f/0x416193e012b0417e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.328125:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d711201f/0x416193e012b0417e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762809.328170:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad82680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015007488:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3212us (47948us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762809.328177:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 672 00000100:00100000:2.0:1789762809.328185:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332628728558 time 332628836342 state 0x2 ref 0 00000100:00100000:2.0:1789762809.330083:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.330133:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015007616 00000100:00100000:1.0:1789762809.330142:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015007616 00000100:00100000:1.0:1789762809.330177:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332628728558 time 332628836342 state 0x0 ref 1 00000100:00100000:1.0:1789762809.330188:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332678836342 time 332628836342 state 0x1 ref 1 00000100:00100000:0.0:1789762809.375016:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332678836342 time 332678912636 state 0x0 ref 1 00000100:00100000:0.0:1789762809.375041:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015007616:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762809.375110:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003f7d9b2d/0x416193e012b04185 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.375123:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762809.375127:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762809.375132:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003f7d9b2d/0x416193e012b04185 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.375143:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003f7d9b2d/0x416193e012b04185 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.378024:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003f7d9b2d/0x416193e012b04185 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.378039:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003f7d9b2d/0x416193e012b04185 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.378050:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003f7d9b2d/0x416193e012b04185 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.378059:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc6aee00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762809.378065:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003f7d9b2d/0x416193e012b04185 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.378075:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003f7d9b2d/0x416193e012b04185 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.378106:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003f7d9b2d/0x416193e012b04185 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.378112:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003f7d9b2d/0x416193e012b04185 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762809.378158:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015007616:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3127us (48075us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762809.378164:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 673 00000100:00100000:0.0:1789762809.378178:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332678836342 time 332678912636 state 0x2 ref 0 00000100:00100000:0.0:1789762809.380368:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762809.380409:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015007744 00000100:00100000:2.0:1789762809.380416:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015007744 00000100:00100000:2.0:1789762809.380451:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332678836342 time 332678912636 state 0x0 ref 1 00000100:00100000:2.0:1789762809.380462:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332728912636 time 332678912636 state 0x1 ref 1 00000100:00100000:1.0:1789762809.425122:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332728912636 time 332729018046 state 0x0 ref 1 00000100:00100000:1.0:1789762809.425144:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015007744:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762809.425179:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0418c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.425192:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.425196:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.425200:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0418c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.425211:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0418c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.428055:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0418c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.428066:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0418c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.428074:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0418c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.428080:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762809.428084:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0418c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.428109:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0418c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.428125:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0418c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.428131:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0418c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762809.428178:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015007744:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3041us (47810us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762809.428184:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 674 00000100:00100000:1.0:1789762809.428192:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332728912636 time 332729018046 state 0x2 ref 0 00000100:00100000:3.0:1789762809.430059:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.430107:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015007872 00000100:00100000:1.0:1789762809.430114:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015007872 00000100:00100000:1.0:1789762809.430148:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332728912636 time 332729018046 state 0x0 ref 1 00000100:00100000:1.0:1789762809.430158:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332779018046 time 332729018046 state 0x1 ref 1 00000100:00100000:2.0:1789762809.475164:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332779018046 time 332779061887 state 0x0 ref 1 00000100:00100000:2.0:1789762809.475184:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86006a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015007872:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762809.475218:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2799d9d/0x416193e012b04193 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.475233:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.475236:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.475241:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2799d9d/0x416193e012b04193 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.475260:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2799d9d/0x416193e012b04193 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.478472:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2799d9d/0x416193e012b04193 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.478485:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2799d9d/0x416193e012b04193 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.478493:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2799d9d/0x416193e012b04193 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.478503:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebff43e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762809.478506:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2799d9d/0x416193e012b04193 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.478513:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2799d9d/0x416193e012b04193 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.478528:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2799d9d/0x416193e012b04193 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.478533:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2799d9d/0x416193e012b04193 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762809.478591:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86006a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015007872:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3407us (48527us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762809.478598:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 675 00000100:00100000:1.0:1789762809.478607:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332779018046 time 332779061887 state 0x2 ref 0 00000100:00100000:3.0:1789762809.480968:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.481013:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015008000 00000100:00100000:1.0:1789762809.481021:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015008000 00000100:00100000:1.0:1789762809.481069:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332779018046 time 332779061887 state 0x0 ref 1 00000100:00100000:1.0:1789762809.481083:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332829061887 time 332779061887 state 0x1 ref 1 00000100:00100000:0.0:1789762809.525285:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332829061887 time 332829180659 state 0x0 ref 1 00000100:00100000:0.0:1789762809.525322:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015008000:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762809.525359:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0419a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.525373:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762809.525376:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762809.525380:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0419a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.525392:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0419a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.528375:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0419a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.528392:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0419a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.528404:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0419a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.528413:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ae00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762809.528419:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0419a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.528429:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0419a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.528448:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0419a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.528458:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0419a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762809.528536:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86006680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015008000:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3222us (47570us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762809.528544:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 676 00000100:00100000:2.0:1789762809.528553:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332829061887 time 332829180659 state 0x2 ref 0 00000100:00100000:3.0:1789762809.530543:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762809.530582:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015008128 00000100:00100000:2.0:1789762809.530591:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015008128 00000100:00100000:2.0:1789762809.530629:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332829061887 time 332829180659 state 0x0 ref 1 00000100:00100000:2.0:1789762809.530640:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332879180659 time 332829180659 state 0x1 ref 1 00000100:00100000:1.0:1789762809.575394:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332879180659 time 332879290370 state 0x0 ref 1 00000100:00100000:1.0:1789762809.575416:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015008128:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762809.575453:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.575467:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.575471:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.575475:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.575486:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.578680:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.578694:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.578701:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.578706:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762809.578710:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.578716:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.578731:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.578737:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762809.578787:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015008128:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3378us (48245us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762809.578793:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 677 00000100:00100000:1.0:1789762809.578801:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332879180659 time 332879290370 state 0x2 ref 0 00000100:00100000:0.0:1789762809.580965:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.581007:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015008256 00000100:00100000:1.0:1789762809.581015:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015008256 00000100:00100000:1.0:1789762809.581048:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332879180659 time 332879290370 state 0x0 ref 1 00000100:00100000:1.0:1789762809.581059:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332929290370 time 332879290370 state 0x1 ref 1 00000100:00100000:1.0:1789762809.625441:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332929290370 time 332929339507 state 0x0 ref 1 00000100:00100000:1.0:1789762809.625460:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015008256:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762809.625490:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.625502:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.625506:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.625510:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.625520:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.628315:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.628326:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.628334:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.628339:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762809.628343:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.628350:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.628375:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.628381:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041a8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762809.628430:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015008256:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2973us (47463us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762809.628439:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 678 00000100:00100000:3.0:1789762809.628463:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332929290370 time 332929339507 state 0x2 ref 0 00000100:00100000:0.0:1789762809.630283:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.630339:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015008384 00000100:00100000:1.0:1789762809.630346:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015008384 00000100:00100000:1.0:1789762809.630378:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332929290370 time 332929339507 state 0x0 ref 1 00000100:00100000:1.0:1789762809.630388:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332979339507 time 332929339507 state 0x1 ref 1 00000100:00100000:2.0:1789762809.675557:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332979339507 time 332979451052 state 0x0 ref 1 00000100:00100000:2.0:1789762809.675585:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015008384:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762809.675626:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b041af lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.675641:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.675644:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.675649:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b041af lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.675674:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b041af lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.679115:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b041af lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.679130:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b041af lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.679140:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b041af lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.679149:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c6f200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762809.679154:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b041af lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.679164:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b041af lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.679182:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b041af lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.679191:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000acea14bc/0x416193e012b041af lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762809.679249:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015008384:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3672us (48965us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762809.679257:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 679 00000100:00100000:2.0:1789762809.679272:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332979339507 time 332979451052 state 0x2 ref 0 00000100:00100000:2.0:1789762809.681562:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762809.681600:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015008512 00000100:00100000:3.0:1789762809.681609:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015008512 00000100:00100000:3.0:1789762809.681647:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 332979339507 time 332979451052 state 0x0 ref 1 00000100:00100000:3.0:1789762809.681658:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333029451052 time 332979451052 state 0x1 ref 1 00000100:00100000:1.0:1789762809.725626:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333029451052 time 333029523274 state 0x0 ref 1 00000100:00100000:1.0:1789762809.725647:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015008512:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762809.725683:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041b6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.725697:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762809.725701:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.725705:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041b6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.725718:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041b6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.728614:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041b6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.728623:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041b6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.728629:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041b6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.728636:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762809.728640:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041b6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.728646:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041b6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.728661:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041b6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.728669:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041b6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762809.728728:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015008512:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3086us (47165us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762809.728738:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 680 00000100:00100000:1.0:1789762809.728749:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333029451052 time 333029523274 state 0x2 ref 0 00000100:00100000:0.0:1789762809.730854:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.730897:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015008640 00000100:00100000:1.0:1789762809.730906:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015008640 00000100:00100000:1.0:1789762809.730939:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333029451052 time 333029523274 state 0x0 ref 1 00000100:00100000:1.0:1789762809.730949:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333079523274 time 333029523274 state 0x1 ref 1 00000100:00100000:3.0:1789762809.775760:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333079523274 time 333079651788 state 0x0 ref 1 00000100:00100000:3.0:1789762809.775788:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015008640:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762809.775824:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041bd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.775838:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762809.775854:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762809.775866:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041bd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.775878:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041bd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.779390:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041bd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.779399:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041bd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.779405:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041bd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.779410:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762809.779414:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041bd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.779420:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041bd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.779432:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041bd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.779437:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041bd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762809.779489:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015008640:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3713us (48635us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762809.779497:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 681 00000100:00100000:3.0:1789762809.779504:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333079523274 time 333079651788 state 0x2 ref 0 00000100:00100000:2.0:1789762809.781459:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.781516:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015008768 00000100:00100000:1.0:1789762809.781524:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015008768 00000100:00100000:1.0:1789762809.781558:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333079523274 time 333079651788 state 0x0 ref 1 00000100:00100000:1.0:1789762809.781589:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333129651788 time 333079651788 state 0x1 ref 1 00010000:00010000:2.0:1789762809.783230:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762809.825865:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333129651788 time 333129755126 state 0x0 ref 1 00000100:00100000:2.0:1789762809.825893:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015008768:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762809.825937:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d41300c/0x416193e012b041c4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.825952:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.825957:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.825963:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d41300c/0x416193e012b041c4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.825978:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d41300c/0x416193e012b041c4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.829380:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d41300c/0x416193e012b041c4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.829399:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d41300c/0x416193e012b041c4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.829408:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d41300c/0x416193e012b041c4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.829422:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec05400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762809.829427:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d41300c/0x416193e012b041c4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.829434:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d41300c/0x416193e012b041c4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.829449:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d41300c/0x416193e012b041c4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.829455:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d41300c/0x416193e012b041c4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762809.829508:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015008768:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3623us (48047us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762809.829516:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 682 00000100:00100000:2.0:1789762809.829526:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333129651788 time 333129755126 state 0x2 ref 0 00000100:00100000:2.0:1789762809.831544:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762809.831583:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015008896 00000100:00100000:3.0:1789762809.831592:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015008896 00000100:00100000:3.0:1789762809.831633:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333129651788 time 333129755126 state 0x0 ref 1 00000100:00100000:3.0:1789762809.831644:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333179755126 time 333129755126 state 0x1 ref 1 00000100:00100000:1.0:1789762809.875942:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333179755126 time 333179836918 state 0x0 ref 1 00000100:00100000:1.0:1789762809.875963:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015008896:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762809.875999:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041cb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.876012:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.876016:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762809.876021:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041cb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.876032:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041cb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.879475:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041cb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.879489:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041cb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.879499:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041cb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.879508:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762809.879512:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041cb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.879519:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041cb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.879531:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041cb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762809.879537:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041cb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762809.879586:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015008896:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3629us (48042us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762809.879593:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 683 00000100:00100000:1.0:1789762809.879602:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333179755126 time 333179836918 state 0x2 ref 0 00000100:00100000:0.0:1789762809.882650:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.882696:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015009024 00000100:00100000:1.0:1789762809.882704:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015009024 00000100:00100000:1.0:1789762809.882774:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333179755126 time 333179836918 state 0x0 ref 1 00000100:00100000:1.0:1789762809.882787:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333229836918 time 333179836918 state 0x1 ref 1 00000100:00100000:3.0:1789762809.926031:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333229836918 time 333229928609 state 0x0 ref 1 00000100:00100000:3.0:1789762809.926053:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015009024:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762809.926124:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041d2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.926142:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762809.926146:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762809.926151:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041d2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.926163:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041d2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.929504:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041d2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.929515:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041d2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.929523:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041d2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.929529:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762809.929533:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041d2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.929539:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041d2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.929553:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041d2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762809.929558:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b041d2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762809.929603:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015009024:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3557us (46954us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762809.929611:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 684 00000100:00100000:3.0:1789762809.929625:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333229836918 time 333229928609 state 0x2 ref 0 00000100:00100000:2.0:1789762809.931654:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762809.931706:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015009152 00000100:00100000:1.0:1789762809.931714:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015009152 00000100:00100000:1.0:1789762809.931747:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333229836918 time 333229928609 state 0x0 ref 1 00000100:00100000:1.0:1789762809.931757:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333279928609 time 333229928609 state 0x1 ref 1 00000100:00100000:2.0:1789762809.976164:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333279928609 time 333280061155 state 0x0 ref 1 00000100:00100000:2.0:1789762809.976185:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015009152:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762809.976223:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d05e353a/0x416193e012b041d9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.976243:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.976247:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762809.976252:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d05e353a/0x416193e012b041d9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762809.976264:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d05e353a/0x416193e012b041d9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.979443:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d05e353a/0x416193e012b041d9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.979454:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d05e353a/0x416193e012b041d9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.979471:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d05e353a/0x416193e012b041d9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.979477:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc36600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762809.979481:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d05e353a/0x416193e012b041d9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.979487:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d05e353a/0x416193e012b041d9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.979502:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d05e353a/0x416193e012b041d9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762809.979508:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d05e353a/0x416193e012b041d9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762809.979563:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015009152:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3381us (47907us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762809.979574:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 685 00000100:00100000:0.0:1789762809.979585:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333279928609 time 333280061155 state 0x2 ref 0 00000100:00100000:3.0:1789762809.982196:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762809.982326:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015009280 00000100:00100000:3.0:1789762809.982338:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015009280 00000100:00100000:3.0:1789762809.982403:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333279928609 time 333280061155 state 0x0 ref 1 00000100:00100000:3.0:1789762809.982429:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333330061155 time 333280061155 state 0x1 ref 1 00000001:00080000:0.0:1789762809.986383:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967398 is committed 00000100:00100000:1.0:1789762810.026266:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333330061155 time 333330162698 state 0x0 ref 1 00000100:00100000:1.0:1789762810.026305:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015009280:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762810.026342:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041e0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.026356:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762810.026360:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762810.026365:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041e0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.026376:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041e0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.029772:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041e0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.029786:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041e0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.029797:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041e0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.029807:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762810.029812:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041e0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.029824:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041e0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.029840:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041e0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.029846:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041e0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762810.029902:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015009280:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3620us (47824us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762810.029908:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 686 00000100:00100000:1.0:1789762810.029917:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333330061155 time 333330162698 state 0x2 ref 0 00000100:00100000:0.0:1789762810.032310:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762810.032351:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015009408 00000100:00100000:1.0:1789762810.032360:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015009408 00000100:00100000:1.0:1789762810.032395:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333330061155 time 333330162698 state 0x0 ref 1 00000100:00100000:1.0:1789762810.032405:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333380162698 time 333330162698 state 0x1 ref 1 00000100:00100000:3.0:1789762810.076386:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333380162698 time 333380271565 state 0x0 ref 1 00000100:00100000:3.0:1789762810.076418:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015009408:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762810.076474:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041e7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.076497:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762810.076503:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762810.076511:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041e7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.076529:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041e7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.079759:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041e7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.079789:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041e7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.079799:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041e7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.079808:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063b200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762810.079814:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041e7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.079826:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041e7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.079847:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041e7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.079867:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041e7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762810.079937:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015009408:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3527us (47649us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762810.079949:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 687 00000100:00100000:3.0:1789762810.079960:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333380162698 time 333380271565 state 0x2 ref 0 00000100:00100000:3.0:1789762810.082398:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762810.082465:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015009536 00000100:00100000:1.0:1789762810.082475:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015009536 00000100:00100000:1.0:1789762810.082522:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333380162698 time 333380271565 state 0x0 ref 1 00000100:00100000:1.0:1789762810.082536:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333430271565 time 333380271565 state 0x1 ref 1 00000100:00100000:0.0:1789762810.126494:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333430271565 time 333430386101 state 0x0 ref 1 00000100:00100000:0.0:1789762810.126563:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015009536:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762810.126599:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b041ee lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.126613:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762810.126617:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762810.126621:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b041ee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.126632:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b041ee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.130484:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b041ee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.130503:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b041ee lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.130511:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b041ee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.130521:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762810.130528:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b041ee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.130541:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b041ee lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.130568:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b041ee lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.130587:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b041ee lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762810.130671:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015009536:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4111us (48271us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762810.130680:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 688 00000100:00100000:2.0:1789762810.130691:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333430271565 time 333430386101 state 0x2 ref 0 00000100:00100000:3.0:1789762810.132676:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762810.132729:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015009664 00000100:00100000:0.0:1789762810.132738:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015009664 00000100:00100000:0.0:1789762810.132789:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333430271565 time 333430386101 state 0x0 ref 1 00000100:00100000:0.0:1789762810.132815:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333480386101 time 333430386101 state 0x1 ref 1 00000100:00100000:1.0:1789762810.176564:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333480386101 time 333480460675 state 0x0 ref 1 00000100:00100000:1.0:1789762810.176584:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015009664:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762810.176619:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041f5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.176633:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762810.176637:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762810.176642:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041f5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.176653:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041f5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.179888:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041f5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.179897:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041f5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.179905:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041f5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.179911:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762810.179915:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041f5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.179923:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041f5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.179940:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041f5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.179949:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b041f5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762810.180002:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015009664:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3424us (47331us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762810.180010:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 689 00000100:00100000:1.0:1789762810.180019:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333480386101 time 333480460675 state 0x2 ref 0 00000100:00100000:0.0:1789762810.181831:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762810.181866:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015009792 00000100:00100000:1.0:1789762810.181874:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015009792 00000100:00100000:1.0:1789762810.181905:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333480386101 time 333480460675 state 0x0 ref 1 00000100:00100000:1.0:1789762810.181918:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333530460675 time 333480460675 state 0x1 ref 1 00000100:00100000:3.0:1789762810.226652:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333530460675 time 333530544958 state 0x0 ref 1 00000100:00100000:3.0:1789762810.226677:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015009792:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762810.226715:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041fc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.226732:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762810.226736:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762810.226743:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.226758:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.230440:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.230454:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041fc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.230465:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.230474:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063b200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762810.230480:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.230498:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041fc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.230517:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041fc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.230527:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005af12251/0x416193e012b041fc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762810.230588:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015009792:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3916us (48756us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762810.230598:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 690 00000100:00100000:3.0:1789762810.230609:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333530460675 time 333530544958 state 0x2 ref 0 00000100:00100000:0.0:1789762810.232545:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762810.232587:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015009920 00000100:00100000:1.0:1789762810.232595:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015009920 00000100:00100000:1.0:1789762810.232632:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333530460675 time 333530544958 state 0x0 ref 1 00000100:00100000:1.0:1789762810.232642:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333580544958 time 333530544958 state 0x1 ref 1 00000100:00100000:2.0:1789762810.276772:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333580544958 time 333580663195 state 0x0 ref 1 00000100:00100000:2.0:1789762810.276801:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015009920:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762810.276850:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04203 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.276867:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762810.276871:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762810.276877:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04203 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.276890:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04203 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.279854:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04203 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.279893:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04203 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.279904:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04203 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.279914:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ae00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762810.279920:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04203 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.279932:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04203 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.279952:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04203 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.279958:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04203 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762810.280021:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015009920:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3231us (47477us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762810.280029:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 691 00000100:00100000:2.0:1789762810.280037:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333580544958 time 333580663195 state 0x2 ref 0 00000100:00100000:2.0:1789762810.281973:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762810.282029:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015010048 00000100:00100000:0.0:1789762810.282041:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015010048 00000100:00100000:0.0:1789762810.282115:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333580544958 time 333580663195 state 0x0 ref 1 00000100:00100000:0.0:1789762810.282127:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333630663195 time 333580663195 state 0x1 ref 1 00000100:00100000:1.0:1789762810.326883:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333630663195 time 333630778521 state 0x0 ref 1 00000100:00100000:1.0:1789762810.326909:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015010048:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762810.326953:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0420a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.326973:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762810.326977:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762810.326985:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0420a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.327000:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0420a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.330358:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0420a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.330372:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0420a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.330382:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0420a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.330391:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762810.330397:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0420a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.330408:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0420a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.330427:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0420a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.330437:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0420a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762810.330493:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015010048:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3593us (48519us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762810.330503:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 692 00000100:00100000:1.0:1789762810.330515:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333630663195 time 333630778521 state 0x2 ref 0 00000100:00100000:2.0:1789762810.357755:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762810.357864:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015010176 00000100:00100000:1.0:1789762810.357879:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015010176 00000100:00100000:1.0:1789762810.357946:0:6664:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 22 00000100:00100000:1.0:1789762810.357955:0:6664:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:5593:x1876702015010176:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762810.357968:0:6664:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762810.358005:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b03f40 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5d7 expref: 6 pid: 6647 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.358027:0:6664:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762810.358075:0:6664:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:5593:x1876702015010176:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 122us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762810.358080:0:6664:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 22 00000100:00100000:0.0:1789762810.367173:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762810.367222:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015010304 00000100:00100000:1.0:1789762810.367228:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015010304 00000100:00080000:3.0:1789762810.367293:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762810.367302:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 262 00000100:00100000:1.0:1789762810.367314:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:12244:x1876702015010304:12345-192.168.203.48@tcp:101:bash.0 00010000:00010000:1.0:1789762810.367331:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762810.367375:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04211 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5de expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.367452:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b04218 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.367466:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b04218 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.367479:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b04218 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762810.367507:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x29:0x0]: returning size 16384 00000004:00200000:1.0:1789762810.367510:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762810.367524:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b04218 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.367542:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04211 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5de expref: 11 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.367558:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b04218 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x29:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5de expref: 10 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.367567:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981e9272ce00) 00010000:00010000:1.0:1789762810.367570:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762810.367573:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81483600, rc 0) 00000100:00100000:1.0:1789762810.367616:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:12244:x1876702015010304:12345-192.168.203.48@tcp:101:bash.0 Request processed in 305us (444us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762810.367624:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 262 00000100:00100000:0.0:1789762810.371359:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762810.371418:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015010560 00000100:00100000:2.0:1789762810.371424:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015010560 00000100:00100000:2.0:1789762810.371458:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 263 00000100:00100000:2.0:1789762810.371467:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:13731:x1876702015010560:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:2.0:1789762810.371475:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762810.371485:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b0421f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5e5 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762810.371527:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x29:0x0]: returning size 16384 00010000:00010000:2.0:1789762810.371647:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b0421f lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5e5 expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.371655:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:2.0:1789762810.371657:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762810.371660:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b0421f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5e5 expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762810.371672:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87c1a400, rc 0) 00000100:00100000:2.0:1789762810.371701:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13731:x1876702015010560:12345-192.168.203.48@tcp:101:rm.0 Request processed in 237us (343us total) trans 4294967516 rc 301/301 00000100:00100000:2.0:1789762810.371706:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 263 00000100:00100000:2.0:1789762810.373188:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762810.373222:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015010688 00000100:00100000:3.0:1789762810.373228:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015010688 00000100:00100000:3.0:1789762810.373256:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 80 00000100:00100000:3.0:1789762810.373262:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13731:x1876702015010688:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:3.0:1789762810.375076:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13731:x1876702015010688:12345-192.168.203.48@tcp:37:rm.0 Request processed in 1813us (1889us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762810.375081:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 80 00000100:00100000:0.0:1789762810.376426:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762810.376466:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015010944 00000100:00100000:3.0:1789762810.376471:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015010944 00000100:00100000:3.0:1789762810.376495:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 81 00000100:00100000:3.0:1789762810.376508:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:13731:x1876702015010944:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762810.376600:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:13731:x1876702015010944:12345-192.168.203.48@tcp:35:rm.0 Request processed in 93us (177us total) trans 4294967517 rc 0/0 00000100:00100000:3.0:1789762810.376606:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 81 00000100:00100000:2.0:1789762810.377413:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762810.377454:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015011200 00000100:00100000:1.0:1789762810.377461:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015011200 00000100:00100000:1.0:1789762810.377497:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 264 00000100:00100000:1.0:1789762810.377508:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81396300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:13731:x1876702015011200:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:1.0:1789762810.377519:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762810.377527:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b0422d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5ec expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.377566:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b04234 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.377574:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b04234 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x29:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.377579:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b04234 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x29:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762810.377585:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x29:0x0]: returning size 16384 00010000:00010000:1.0:1789762810.377629:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b0422d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5ec expref: 15 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.377639:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b04234 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x29:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5ec expref: 14 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.377645:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981e9272ce00) 00010000:00010000:1.0:1789762810.377647:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1789762810.377649:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81483400, rc 0) 00000100:00100000:1.0:1789762810.377680:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81396300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13731:x1876702015011200:12345-192.168.203.48@tcp:101:rm.0 Request processed in 176us (267us total) trans 4294967518 rc 0/0 00000100:00100000:1.0:1789762810.377688:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 264 00000100:00100000:0.0:1789762810.378553:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762810.378587:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015011328 00000100:00100000:1.0:1789762810.378592:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015011328 00000100:00100000:1.0:1789762810.378616:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 265 00000100:00100000:1.0:1789762810.378622:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13731:x1876702015011328:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:1.0:1789762810.378652:0:8165:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7597981696 avail 7586447360 00000100:00100000:1.0:1789762810.378683:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13731:x1876702015011328:12345-192.168.203.48@tcp:41:rm.0 Request processed in 62us (131us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762810.378689:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 265 00000100:00100000:0.0:1789762810.379438:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762810.379495:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015011584 00000100:00100000:3.0:1789762810.379509:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015011584 00000100:00100000:3.0:1789762810.379577:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 82 00000100:00100000:3.0:1789762810.379590:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b1ac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13731:x1876702015011584:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762810.379880:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b1ac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:13731:x1876702015011584:12345-192.168.203.48@tcp:35:rm.0 Request processed in 290us (438us total) trans 4294967519 rc 0/0 00000100:00100000:3.0:1789762810.379893:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 82 00000100:00100000:2.0:1789762810.380544:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762810.380587:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015011840 00000100:00100000:0.0:1789762810.380593:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015011840 00000100:00100000:0.0:1789762810.380621:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 266 00000100:00100000:0.0:1789762810.380628:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81397480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:13731:x1876702015011840:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:0.0:1789762810.380653:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1789762810.380665:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b03f32 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5d0 expref: 15 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.380679:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b03c53 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5c9 expref: 14 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.380687:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff981ea0aa3c00) 00010000:00010000:0.0:1789762810.380695:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b04218 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x29:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5de expref: 13 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.380701:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981e9272ce00) 00010000:00010000:0.0:1789762810.380703:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762810.380713:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b04242 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.380723:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b04242 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.380745:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b04242 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.380782:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04249 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.380789:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04249 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.380794:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04249 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.381078:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04250 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.381106:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04250 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.381113:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04250 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762810.382006:0:5806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].17, 64 off9280 00000004:00080000:0.0:1789762810.382015:0:5806:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 333685927314ns, next at 334685925621ns, rc = 0 00010000:00010000:0.0:1789762810.382052:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b04257 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382059:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b04257 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382065:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b04257 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382078:0:5806:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b04257 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382083:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b04257 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382104:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b04257 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382119:0:5806:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762810.382132:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04250 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382138:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04250 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382143:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04250 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382149:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762810.382153:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762810.382154:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04250 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.382164:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b04257 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382165:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04250 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.382188:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762810.382189:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762810.382193:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762810.382200:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04250 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.382200:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b04257 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.382231:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b04257 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382234:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04250 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2b:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.382255:0:8466:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:1.0:1789762810.382259:0:8466:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762810.382265:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b04242 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.382265:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b04257 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.382280:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf800/0x416193e012b04257 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382294:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b04242 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382303:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b04242 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382309:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762810.382309:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762810.382314:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762810.382315:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b04242 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.382318:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762810.382327:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b04242 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382340:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981e9272ce00) 00010000:00010000:0.0:1789762810.382343:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762810.382346:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b04242 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762810.382352:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b04242 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762810.382362:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000050ce5374 reply state 0000000010f032b7 transno 4294967520 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04249 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762810.382371:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762810.382430:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81397480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13731:x1876702015011840:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1800us (1886us total) trans 4294967520 rc 0/0 00000100:00100000:0.0:1789762810.382438:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 266 00010000:00010000:3.0:1789762810.382986:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04249 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.382997:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04249 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.383003:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04249 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.383008:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbfc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762810.383011:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04249 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.383016:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04249 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.383029:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x4009f790 (ffff981ea0aa3a00) 00010000:00010000:3.0:1789762810.383043:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762810.383047:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04249 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.383052:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04249 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x29:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762810.383489:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762810.383531:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015012096 00000100:00100000:2.0:1789762810.383537:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015012096 00000100:00100000:2.0:1789762810.383569:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 83 00000100:00100000:2.0:1789762810.383583:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:13731:x1876702015012096:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:2.0:1789762810.383675:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13731:x1876702015012096:12345-192.168.203.48@tcp:35:rm.0 Request processed in 93us (188us total) trans 4294967521 rc 0/0 00000100:00100000:2.0:1789762810.383681:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 83 00000100:00100000:0.0:1789762810.384382:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762810.384412:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015012352 00000100:00100000:1.0:1789762810.384417:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015012352 00000100:00100000:1.0:1789762810.384441:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 267 00000100:00100000:1.0:1789762810.384447:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:13731:x1876702015012352:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:1.0:1789762810.384466:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1789762810.384476:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b04234 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x29:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5ec expref: 14 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.384490:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b03c37 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x29:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5c2 expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.384501:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981e9272ce00) 00010000:00010000:1.0:1789762810.384514:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b03bd5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5ad expref: 12 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.384524:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:1.0:1789762810.384531:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762810.384542:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0425e lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.384550:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0425e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.384559:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0425e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.384571:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b04265 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.384576:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b04265 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.384581:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b04265 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.384769:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b0426c lrc: 3/0,1 mode: --/EX res: [0x200000401:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.384776:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b0426c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x29:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.384781:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b0426c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x29:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762810.385166:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x29:0x0]: returning size 16384 00010000:00010000:1.0:1789762810.385174:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b0426c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x29:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385179:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b0426c lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x29:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385184:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b0426c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x29:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385187:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762810.385189:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b0426c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x29:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385194:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b0426c lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x29:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385199:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x29:0x0].0x0 (ffff981ec2179700) 00010000:00010000:1.0:1789762810.385201:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762810.385204:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b0426c lrc: 2/0,0 mode: --/EX res: [0x200000401:0x29:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385208:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b0426c lrc: 0/0,0 mode: --/EX res: [0x200000401:0x29:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385218:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0425e lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385223:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0425e lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385227:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0425e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385230:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762810.385232:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0425e lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385235:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0425e lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385239:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:1.0:1789762810.385241:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762810.385242:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0425e lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762810.385246:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483c00/0x416193e012b0425e lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762810.385251:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009bc28a7d reply state 00000000bce18fe7 transno 4294967522 ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b04265 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762810.385256:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762810.385321:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:13731:x1876702015012352:12345-192.168.203.48@tcp:36:rm.0 Request processed in 873us (939us total) trans 4294967522 rc 0/0 00000100:00100000:1.0:1789762810.385327:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 267 00010000:00010000:3.0:1789762810.385938:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b04265 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.385950:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b04265 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.385957:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b04265 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.385963:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762810.385968:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b04265 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.385975:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b04265 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.385986:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179100) 00010000:00010000:3.0:1789762810.385989:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762810.385993:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b04265 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762810.386001:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b04265 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762810.498117:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967910 is committed 00010000:00010000:0.0:1789762810.807365:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1789762811.284358:0:15078:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 334588268640ns, next at 334685925621ns, rc = 0 00000001:00080000:1.0:1789762811.293229:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967520 is committed 00000004:00080000:1.0:1789762811.293237:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 21 committed 00000001:00080000:1.0:1789762811.293252:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967522 is committed 00000004:00080000:1.0:1789762811.293261:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 22 committed 00000100:00100000:1.0:1789762811.293371:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81394700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022167552:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789762811.293428:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762811.293455:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022167552 00000100:00100000:0.0:1789762811.293462:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022167552 00000100:00080000:3.0:1789762811.293497:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762811.293525:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:0.0:1789762811.293536:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb494d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022167552:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1789762811.293562:0:6881:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:17 count 1 00002000:00010000:0.0:1789762811.293588:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 104857600, mtime: 0x6aad9cf5, atime: 0x0, ctime: 0x6aad9cf5, blocks: 0x3200b 00010000:00010000:0.0:1789762811.293595:0:6881:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04273 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762811.293609:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=334s v=(0 0) 00010000:00010000:0.0:1789762811.293612:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762811.293617:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04273 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762811.293629:0:6881:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04273 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762811.293636:0:6881:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04273 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762811.293642:0:6881:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04273 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762811.293666:0:6881:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762811.293701:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762811.293708:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04273 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762811.293718:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762811.293723:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04273 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762811.293732:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04273 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762811.293749:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04273 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762811.293756:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04273 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762811.293771:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762811.293773:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762811.293775:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762811.294068:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb494d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022167552:12345-0@lo:6:osp-syn-0-0.0 Request processed in 535us (638us total) trans 4294967911 rc 0/0 00000100:00100000:0.0:1789762811.294075:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:1.0:1789762811.294122:0:3274:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967910 gen 2 00000004:00080000:1.0:1789762811.294127:0:3274:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebc554000, transno 4294967910 00000100:00100000:1.0:1789762811.294140:0:3274:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff981d81394700 x1876702022167552/t4294967911(4294967911) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789762827 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:1.0:1789762811.294149:0:3274:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981d81394700/2, rc 0, transno 4294967911 00000100:00100000:1.0:1789762811.294160:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81394700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022167552:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1789762811.294345:0:15078:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 334598256291ns, next at 327623192251ns, rc = 0 00000004:00080000:1.0:1789762811.300205:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 18 committed 00010000:00010000:2.0:1789762811.831288:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762812.855236:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762813.026281:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762813.026321:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015012608 00000100:00100000:3.0:1789762813.026329:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762813.026333:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015012608 00000100:00100000:3.0:1789762813.026362:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762813.026394:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762813.026404:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 248 00000100:00100000:1.0:1789762813.026416:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015012608:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762813.026429:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762813.026459:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015012480 00000100:00100000:3.0:1789762813.026463:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015012480 00000100:00100000:1.0:1789762813.026489:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015012608:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 75us (225us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762813.026496:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 248 00000100:00100000:3.0:1789762813.026509:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 304 00000100:00100000:1.0:1789762813.026518:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015012992 00000100:00100000:3.0:1789762813.026520:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015012480:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762813.026522:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015012992 00000100:00100000:1.0:1789762813.026559:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 249 00000100:00100000:3.0:1789762813.026561:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015012480:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 42us (234us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762813.026573:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 304 00000100:00100000:1.0:1789762813.026574:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015012992:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762813.026609:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015012992:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 41us (180us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762813.026609:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762813.026622:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 249 00000100:00100000:3.0:1789762813.026629:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015012736 00000100:00100000:3.0:1789762813.026634:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015012736 00000100:00100000:3.0:1789762813.026679:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015012864 00000100:00100000:3.0:1789762813.026681:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015012864 00000100:00100000:1.0:1789762813.026691:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 123 00000100:00100000:1.0:1789762813.026704:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015012736:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762813.026716:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 124 00000100:00100000:3.0:1789762813.026729:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015012864:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762813.026754:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015012736:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 43us (379us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762813.026763:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 123 00000100:00100000:3.0:1789762813.026771:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015012864:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 44us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762813.026783:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 124 00000100:00080000:3.0:1789762813.026812:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789762813.026816:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762813.047240:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc556680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022167680:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762813.047318:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762813.047366:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022167680 00000100:00100000:0.0:1789762813.047374:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022167680 00000100:00100000:0.0:1789762813.047424:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 105 00000100:00080000:2.0:1789762813.047424:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762813.047436:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022167680:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762813.047466:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762813.047508:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022167680:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (192us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762813.047514:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 105 00000100:00100000:1.0:1789762813.047545:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc556680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022167680:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762813.559318:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4ab4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022167936:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762813.559319:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86007b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022167808:0@lo:400:kworker.0 00000100:00100000:0.0:1789762813.559340:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86007800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022168192:0@lo:400:kworker.0 00000100:00100000:3.0:1789762813.559366:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86006d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022168064:0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.559413:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762813.559424:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762813.559436:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762813.559471:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022167808 00000100:00100000:3.0:1789762813.559481:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022167808 00000100:00100000:1.0:1789762813.559516:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86004380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022168320:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.559521:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86005500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022168448:0@lo:400:kworker.0 00000100:00100000:0.0:1789762813.559552:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022167936 00000100:00100000:3.0:1789762813.559559:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 305 00000100:00100000:0.0:1789762813.559560:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022167936 00000100:00100000:1.0:1789762813.559574:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762813.559587:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762813.559587:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85086450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022167808:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762813.559641:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 106 00000100:00100000:1.0:1789762813.559649:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022168192 00000100:00100000:0.0:1789762813.559657:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc556680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022167936:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762813.559660:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022168192 00000100:00080000:2.0:1789762813.559662:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762813.559682:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85086450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022167808:12345-0@lo:400:kworker.0 Request processed in 109us (262us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762813.559691:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 305 00002000:00000024:0.0:1789762813.559699:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762813.559717:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86007b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022167808:0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.559730:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022168320 00000100:00100000:1.0:1789762813.559735:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022168320 00000100:00080000:2.0:1789762813.559766:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762813.559769:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762813.559774:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc556680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022167936:12345-0@lo:13:osp-pre-1-0.0 Request processed in 123us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762813.559782:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 106 00000100:00100000:1.0:1789762813.559800:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022168064 00000100:00100000:1.0:1789762813.559808:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022168064 00000100:00100000:2.0:1789762813.559817:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022168448 00000100:00100000:2.0:1789762813.559823:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022168448 00000100:00100000:0.0:1789762813.559841:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:0.0:1789762813.559856:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022168192:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762813.559869:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:1.0:1789762813.559883:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:3.0:1789762813.559893:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022168320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762813.559897:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86007b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022168064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.559908:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 125 00000100:00100000:0.0:1789762813.559923:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022168192:12345-0@lo:400:kworker.0 Request processed in 70us (488us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762813.559932:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022168448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762813.559942:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1789762813.559954:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022168320:12345-0@lo:400:kworker.0 Request processed in 74us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762813.559963:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00100000:1.0:1789762813.559985:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86007b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022168064:12345-0@lo:400:kworker.0 Request processed in 92us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762813.559999:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:0.0:1789762813.560026:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86007800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022168192:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.560039:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022168448:12345-0@lo:400:kworker.0 Request processed in 117us (459us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762813.560054:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 125 00000100:00100000:1.0:1789762813.560132:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86006d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022168064:0@lo:400:kworker.0 00000100:00100000:3.0:1789762813.560141:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86004380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022168320:0@lo:400:kworker.0 00000100:00100000:2.0:1789762813.560173:0:3273:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967398 gen 2 00000004:00080000:2.0:1789762813.560179:0:3273:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebfdcb100, transno 4294967398 00000100:00100000:2.0:1789762813.560195:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86005500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022168448:0@lo:400:kworker.0 00000100:00100000:3.0:1789762813.560204:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4ab4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022167936:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789762813.879200:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762814.903300:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762815.621980:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967911 is committed 00010000:00010000:2.0:1789762815.927267:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762816.951215:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762817.975234:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762818.146118:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762818.146160:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015013120 00000100:00100000:1.0:1789762818.146172:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015013120 00000100:00100000:1.0:1789762818.146239:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 306 00000100:00100000:1.0:1789762818.146252:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85085850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015013120:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762818.146324:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85085850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015013120:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 74us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762818.146331:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 306 00000100:00100000:0.0:1789762818.147298:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762818.147335:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015013248 00000100:00100000:1.0:1789762818.147340:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015013248 00000100:00100000:0.0:1789762818.147376:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762818.147392:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762818.147414:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762818.147414:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 253 00000100:00100000:1.0:1789762818.147427:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015013248:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762818.147429:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015013376 00000100:00100000:2.0:1789762818.147435:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015013376 00000100:00100000:0.0:1789762818.147447:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762818.147459:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015013632 00000100:00100000:3.0:1789762818.147466:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015013632 00000100:00100000:1.0:1789762818.147470:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015013248:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 45us (171us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762818.147478:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 253 00000100:00100000:2.0:1789762818.147479:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015013504 00000100:00100000:2.0:1789762818.147484:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015013504 00000100:00100000:3.0:1789762818.147516:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 254 00000100:00100000:2.0:1789762818.147521:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 126 00000100:00100000:3.0:1789762818.147529:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015013632:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762818.147534:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015013376:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762818.147548:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 127 00000100:00100000:0.0:1789762818.147562:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015013504:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762818.147578:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015013376:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 48us (201us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762818.147585:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 126 00000100:00080000:2.0:1789762818.147604:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762818.147605:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015013632:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 80us (157us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762818.147609:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762818.147615:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015013504:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 55us (203us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762818.147615:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762818.147621:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 254 00000100:00100000:0.0:1789762818.147632:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 127 00000100:00100000:1.0:1789762818.167185:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789762818.167203:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc556d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022168576:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762818.167263:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762818.167306:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022168576 00000100:00100000:0.0:1789762818.167313:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022168576 00000100:00080000:2.0:1789762818.167338:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762818.167372:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 107 00000100:00100000:0.0:1789762818.167381:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022168576:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762818.167407:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762818.167445:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022168576:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (183us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762818.167451:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 107 00000100:00100000:1.0:1789762818.167473:0:3273:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967911 gen 2 00000004:00080000:1.0:1789762818.167478:0:3273:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981d81394700, transno 4294967911 00000100:00100000:1.0:1789762818.167496:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc556d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022168576:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762818.679262:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e98e7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022168704:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762818.679262:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc554380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022168960:0@lo:400:kworker.0 00000100:00100000:0.0:1789762818.679266:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc557800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022169216:0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.679276:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc557b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022169088:0@lo:400:kworker.0 00000100:00100000:0.0:1789762818.679343:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762818.679352:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762818.679366:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762818.679384:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022169216 00000100:00100000:3.0:1789762818.679391:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022169216 00000100:00100000:2.0:1789762818.679402:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc556d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022168832:0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.679422:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022169088 00000100:00100000:1.0:1789762818.679428:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022169088 00000100:00080000:0.0:1789762818.679435:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762818.679444:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762818.679451:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:0.0:1789762818.679455:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022168704 00000100:00100000:0.0:1789762818.679462:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022168704 00000100:00100000:3.0:1789762818.679463:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022169216:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762818.679479:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 256 00000100:00100000:1.0:1789762818.679489:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc556a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022169088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762818.679495:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc554e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022169344:0@lo:400:kworker.0 00000100:00100000:0.0:1789762818.679517:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 108 00000100:00100000:3.0:1789762818.679526:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022169216:12345-0@lo:400:kworker.0 Request processed in 66us (182us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762818.679530:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022168704:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762818.679536:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00000100:00100000:2.0:1789762818.679543:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762818.679551:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc556a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022169088:12345-0@lo:400:kworker.0 Request processed in 64us (187us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762818.679559:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 256 00002000:00000024:0.0:1789762818.679560:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762818.679567:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022168832 00000100:00100000:3.0:1789762818.679571:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022168832 00000100:00100000:1.0:1789762818.679605:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc557b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022169088:0@lo:400:kworker.0 00000100:00100000:0.0:1789762818.679611:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022168704:12345-0@lo:13:osp-pre-1-0.0 Request processed in 85us (262us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762818.679619:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762818.679622:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00100000:0.0:1789762818.679624:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 108 00000100:00100000:3.0:1789762818.679638:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85087c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022168832:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789762818.679644:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762818.679660:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022169344 00000100:00100000:0.0:1789762818.679667:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022169344 00000100:00100000:3.0:1789762818.679687:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85087c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022168832:12345-0@lo:400:kworker.0 Request processed in 53us (243us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762818.679695:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e98e7d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022168704:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762818.679696:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc557800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022169216:0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.679698:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00000100:00100000:0.0:1789762818.679707:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 128 00000100:00100000:0.0:1789762818.679717:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81397b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022169344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762818.679728:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc556d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022168832:0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.679748:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022168960 00000100:00100000:3.0:1789762818.679753:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022168960 00000100:00100000:0.0:1789762818.679758:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81397b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022169344:12345-0@lo:400:kworker.0 Request processed in 43us (216us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762818.679764:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 128 00000100:00100000:2.0:1789762818.679772:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc554e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022169344:0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.679789:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 257 00000100:00100000:3.0:1789762818.679797:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81396d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022168960:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762818.679829:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81396d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022168960:12345-0@lo:400:kworker.0 Request processed in 34us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762818.679834:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 257 00000100:00100000:2.0:1789762818.679862:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc554380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022168960:0@lo:400:kworker.0 00010000:00010000:3.0:1789762818.999250:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762820.023250:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762821.047252:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762822.071226:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762823.095272:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762823.266308:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762823.266345:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762823.266357:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015013888 00000100:00100000:3.0:1789762823.266367:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015013888 00000100:00100000:0.0:1789762823.266370:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762823.266401:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762823.266404:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015013760 00000100:00100000:1.0:1789762823.266420:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015013760 00000100:00100000:3.0:1789762823.266435:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 258 00000100:00100000:2.0:1789762823.266440:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015014144 00000100:00100000:2.0:1789762823.266447:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015014144 00000100:00100000:3.0:1789762823.266453:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015013888:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762823.266454:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762823.266472:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 308 00000100:00100000:1.0:1789762823.266485:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85086050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015013760:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762823.266496:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015014016 00000100:00100000:0.0:1789762823.266498:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015014272 00000100:00100000:2.0:1789762823.266500:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015014016 00000100:00100000:0.0:1789762823.266506:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015014272 00000100:00100000:3.0:1789762823.266526:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015013888:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 81us (225us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762823.266533:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85086050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015013760:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 48us (182us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762823.266543:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 258 00000100:00100000:2.0:1789762823.266544:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 129 00000100:00100000:1.0:1789762823.266550:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 308 00000100:00100000:0.0:1789762823.266559:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 259 00000100:00100000:2.0:1789762823.266564:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015014016:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762823.266585:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015014272:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:3.0:1789762823.266595:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762823.266600:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789762823.266602:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762823.266611:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015014016:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 53us (208us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762823.266623:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 129 00000100:00100000:0.0:1789762823.266630:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015014272:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 46us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762823.266634:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 170 00000100:00100000:0.0:1789762823.266659:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 259 00000100:00100000:3.0:1789762823.266659:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2362:x1876702015014144:12345-192.168.203.48@tcp:9:kworker.0 00000020:00000080:3.0:1789762823.266680:0:6649:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x416193e012b03a3f 00000100:00080000:3.0:1789762823.266696:0:6649:0:(import.c:65:import_set_state_nolock()) ffff981d8a644000 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1789762823.266742:0:6649:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967398 00010000:00010000:2.0:1789762823.266750:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762823.267033:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+2:2362:x1876702015014144:12345-192.168.203.48@tcp:9:kworker.0 Request processed in 387us (662us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762823.267043:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 170 00000100:00100000:0.0:1789762823.287243:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789762823.287261:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc557800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022169472:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762823.287308:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762823.287360:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022169472 00000100:00100000:2.0:1789762823.287369:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022169472 00000100:00100000:2.0:1789762823.287429:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 109 00000100:00080000:3.0:1789762823.287442:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762823.287443:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022169472:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762823.287474:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762823.287516:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022169472:12345-0@lo:13:osp-pre-0-0.0 Request processed in 79us (207us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762823.287521:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 109 00000100:00100000:0.0:1789762823.287564:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc557800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022169472:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762823.799263:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81397800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022169600:0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.799283:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81395180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022169856:0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.799297:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81396680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022169728:0@lo:400:kworker.0 00000100:00100000:2.0:1789762823.799311:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022169984:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762823.799337:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762823.799343:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762823.799374:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762823.799374:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762823.799411:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81397100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022170112:0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.799414:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81394e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022170240:0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.799416:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81396a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022170368:0@lo:400:kworker.0 00000100:00100000:2.0:1789762823.799431:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022169600 00000100:00100000:2.0:1789762823.799438:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022169600 00000100:00100000:1.0:1789762823.799447:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762823.799465:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762823.799481:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022169856 00000100:00100000:0.0:1789762823.799486:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022169856 00000100:00100000:2.0:1789762823.799493:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 309 00000100:00100000:2.0:1789762823.799506:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85087050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022169600:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.799526:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022169728 00000100:00100000:3.0:1789762823.799533:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022169728 00000100:00100000:0.0:1789762823.799541:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022170240 00000100:00100000:1.0:1789762823.799542:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022170112 00000100:00100000:0.0:1789762823.799546:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022170240 00000100:00100000:1.0:1789762823.799548:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022170112 00000100:00100000:2.0:1789762823.799569:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85087050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022169600:12345-0@lo:400:kworker.0 Request processed in 65us (231us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762823.799579:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 260 00000100:00100000:0.0:1789762823.799589:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022169856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.799594:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:2.0:1789762823.799595:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 309 00000100:00100000:1.0:1789762823.799597:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:1.0:1789762823.799616:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022170112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.799617:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022169728:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.799643:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022169856:12345-0@lo:400:kworker.0 Request processed in 56us (301us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762823.799650:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 260 00000100:00100000:2.0:1789762823.799660:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:2.0:1789762823.799671:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86007480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022170240:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762823.799678:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762823.799679:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc557b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022170112:12345-0@lo:400:kworker.0 Request processed in 69us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762823.799691:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00100000:3.0:1789762823.799692:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022169728:12345-0@lo:400:kworker.0 Request processed in 81us (323us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762823.799702:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762823.799710:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00100000:2.0:1789762823.799714:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86007480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022170240:12345-0@lo:400:kworker.0 Request processed in 44us (250us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762823.799722:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022170368 00000100:00100000:2.0:1789762823.799723:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00100000:1.0:1789762823.799728:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022170368 00000100:00100000:0.0:1789762823.799746:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81395180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022169856:0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.799766:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81396680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022169728:0@lo:400:kworker.0 00000100:00100000:1.0:1789762823.799773:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00080000:0.0:1789762823.799780:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762823.799784:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022170368:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762823.799790:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81394e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022170240:0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.799809:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022169984 00000100:00100000:0.0:1789762823.799820:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022169984 00000100:00100000:1.0:1789762823.799843:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022170368:12345-0@lo:400:kworker.0 Request processed in 62us (145us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762823.799850:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00100000:0.0:1789762823.799854:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 110 00000100:00100000:0.0:1789762823.799865:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022169984:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762823.799874:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81396a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022170368:0@lo:400:kworker.0 00002000:00000024:0.0:1789762823.799892:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762823.799893:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81397100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022170112:0@lo:400:kworker.0 00000100:00100000:1.0:1789762823.799906:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81397800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022169600:0@lo:400:kworker.0 00000100:00100000:0.0:1789762823.799938:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022169984:12345-0@lo:13:osp-pre-1-0.0 Request processed in 76us (570us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762823.799944:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 110 00000100:00100000:2.0:1789762823.799968:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022169984:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789762824.119266:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762825.143210:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:0.0:1789762825.344899:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:0.0:1789762825.344908:0:7605:0:(genops.c:885:class_export_put()) final put ffff981d8a640000/f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:2.0:1789762825.344946:0:37:0:(genops.c:844:class_export_destroy()) destroying export ffff981d8a640000/f1bd28de-ee78-4d18-bad3-fa7cec968893 for lustre-OST0001 00000020:00000080:2.0:1789762825.344999:0:37:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff981d8a644000 for lustre-OST0001 00010000:00010000:0.0:1789762826.167224:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762827.191252:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762828.215253:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762828.386293:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762828.386324:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762828.386346:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762828.386359:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015014400 00000100:00100000:3.0:1789762828.386373:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015014528 00000100:00100000:1.0:1789762828.386376:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015014400 00000100:00100000:0.0:1789762828.386381:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762828.386383:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015014528 00000100:00100000:0.0:1789762828.386412:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015014656 00000100:00100000:0.0:1789762828.386417:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015014656 00000100:00100000:2.0:1789762828.386420:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015014784 00000100:00100000:2.0:1789762828.386427:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015014784 00000100:00100000:1.0:1789762828.386447:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 310 00000100:00100000:3.0:1789762828.386454:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 263 00000100:00100000:0.0:1789762828.386460:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 132 00000100:00100000:1.0:1789762828.386467:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015014400:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762828.386471:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015014528:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762828.386474:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015014656:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762828.386486:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 264 00000100:00100000:2.0:1789762828.386497:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad82d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015014784:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762828.386539:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015014656:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 70us (160us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762828.386547:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015014400:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 83us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762828.386551:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad82d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015014784:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 54us (201us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762828.386551:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015014528:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 84us (221us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762828.386553:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 132 00000100:00100000:1.0:1789762828.386570:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 310 00000100:00100000:2.0:1789762828.386571:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 264 00000100:00100000:3.0:1789762828.386584:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 263 00000100:00080000:3.0:1789762828.386607:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762828.386612:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762828.407287:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789762828.407307:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc554a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022170496:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762828.407361:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762828.407403:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022170496 00000100:00100000:2.0:1789762828.407412:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022170496 00000100:00080000:3.0:1789762828.407441:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762828.407464:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 111 00000100:00100000:2.0:1789762828.407475:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022170496:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762828.407507:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762828.407556:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad82a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022170496:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762828.407573:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 111 00000100:00100000:0.0:1789762828.407598:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc554a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022170496:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762828.919251:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc554380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022171008:0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.919261:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022170752:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762828.919310:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc557800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022170880:0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.919339:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762828.919341:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762828.919359:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc554e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022171136:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.919393:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762828.919407:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc554a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022170624:0@lo:400:kworker.0 00000100:00100000:0.0:1789762828.919427:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022171008 00000100:00100000:0.0:1789762828.919435:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022171008 00000100:00100000:1.0:1789762828.919435:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022171136 00000100:00100000:1.0:1789762828.919439:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022171136 00000100:00100000:3.0:1789762828.919450:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762828.919495:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 265 00000100:00100000:1.0:1789762828.919495:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 133 00000100:00100000:0.0:1789762828.919510:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad83800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022171008:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.919510:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc556300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022171136:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.919513:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc556d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022171264:0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.919542:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762828.919581:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad83800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022171008:12345-0@lo:400:kworker.0 Request processed in 74us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762828.919581:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc556300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022171136:12345-0@lo:400:kworker.0 Request processed in 75us (185us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762828.919590:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022170624 00000100:00100000:0.0:1789762828.919592:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 265 00000100:00100000:3.0:1789762828.919596:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022170624 00000100:00100000:1.0:1789762828.919600:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 133 00000100:00100000:0.0:1789762828.919616:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022171264 00000100:00100000:0.0:1789762828.919621:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022171264 00000100:00100000:3.0:1789762828.919642:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 311 00000100:00100000:3.0:1789762828.919653:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85087450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022170624:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.919661:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc554e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022171136:0@lo:400:kworker.0 00000100:00100000:0.0:1789762828.919671:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:0.0:1789762828.919681:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86006d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022171264:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789762828.919713:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762828.919713:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85087450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022170624:12345-0@lo:400:kworker.0 Request processed in 61us (264us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762828.919725:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 311 00000100:00100000:0.0:1789762828.919730:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86006d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022171264:12345-0@lo:400:kworker.0 Request processed in 50us (187us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762828.919730:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022170752 00000100:00100000:1.0:1789762828.919734:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022170752 00000100:00100000:0.0:1789762828.919738:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00100000:1.0:1789762828.919779:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 112 00000100:00100000:1.0:1789762828.919790:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022170752:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762828.919798:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc554380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022171008:0@lo:400:kworker.0 00000100:00100000:3.0:1789762828.919819:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc556d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022171264:0@lo:400:kworker.0 00002000:00000024:1.0:1789762828.919823:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762828.919834:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc554a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022170624:0@lo:400:kworker.0 00000100:00100000:0.0:1789762828.919844:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762828.919867:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022170752:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (532us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762828.919875:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad81f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022170752:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762828.919876:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc554000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022171392:0@lo:400:kworker.0 00000100:00100000:1.0:1789762828.919876:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 112 00000100:00100000:2.0:1789762828.919886:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022170880 00000100:00100000:2.0:1789762828.919893:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022170880 00000100:00080000:1.0:1789762828.919902:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762828.919909:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762828.919938:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:2.0:1789762828.919951:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022170880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762828.919953:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022171392 00000100:00100000:0.0:1789762828.919959:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022171392 00000100:00100000:0.0:1789762828.920000:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 134 00000100:00100000:2.0:1789762828.920008:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022170880:12345-0@lo:400:kworker.0 Request processed in 61us (165us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762828.920013:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022171392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762828.920023:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:1.0:1789762828.920051:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc557800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022170880:0@lo:400:kworker.0 00000100:00100000:0.0:1789762828.920062:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022171392:12345-0@lo:400:kworker.0 Request processed in 54us (153us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762828.920070:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 134 00000100:00100000:1.0:1789762828.920076:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc554000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022171392:0@lo:400:kworker.0 00010000:00010000:2.0:1789762829.239366:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762829.718438:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:3.0:1789762829.718448:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:3.0:1789762829.718456:0:15596:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762829.718464:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:3.0:1789762829.718473:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:3.0:1789762829.718478:0:15596:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981e880d3400 rate 20 gen 0 00000100:00100000:3.0:1789762829.718482:0:15596:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762829.718485:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:3.0:1789762829.718489:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:3.0:1789762829.718493:0:15596:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981e880d3f00 rate 20 gen 0 00000100:00100000:3.0:1789762829.718760:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:3.0:1789762829.718866:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:3.0:1789762829.718872:0:15596:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762829.718876:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:3.0:1789762829.718878:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:3.0:1789762829.718881:0:15596:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981e880d3e00 rate 10 gen 0 00000100:00100000:3.0:1789762829.718884:0:15596:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762829.718887:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: jobid, str: dd.500 00000100:00100000:3.0:1789762829.718889:0:15596:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:3.0:1789762829.718892:0:15596:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981e880d3a00 rate 10 gen 0 00000100:00100000:3.0:1789762829.729539:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762829.729585:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015014912 00000100:00100000:0.0:1789762829.729594:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015014912 00000100:00100000:0.0:1789762829.729642:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 268 00000100:00100000:0.0:1789762829.729656:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86007800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:14210:x1876702015014912:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:0.0:1789762829.729673:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762829.729730:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0427a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5f3 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.729783:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b04281 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.729796:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b04281 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.729810:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b04281 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762829.729846:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762829.729850:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762829.729868:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b04281 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.729889:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b0427a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5f3 expref: 7 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.729903:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b04281 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5f3 expref: 6 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762829.729909:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3800) 00010000:00010000:0.0:1789762829.729912:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762829.729922:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbf200, rc 0) 00000100:00100000:0.0:1789762829.729974:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86007800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14210:x1876702015014912:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 323us (440us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762829.729980:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 268 00000100:00100000:1.0:1789762829.731279:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762829.731316:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015015168 00000100:00100000:3.0:1789762829.731322:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015015168 00000100:00100000:3.0:1789762829.731363:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 269 00000100:00100000:3.0:1789762829.731374:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14210:x1876702015015168:12345-192.168.203.48@tcp:36:mkdir.0 00010000:00010000:3.0:1789762829.731401:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762829.731415:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf200/0x416193e012b04281 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5f3 expref: 6 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.731427:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3800) 00010000:00010000:3.0:1789762829.731435:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762829.731649:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04288 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.731674:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04288 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.731684:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04288 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.731725:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b0428f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.731732:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b0428f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.731738:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b0428f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762829.732860:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 16384 00010000:00010000:3.0:1789762829.732869:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04288 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.732875:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04288 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.732880:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04288 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.732884:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063aa00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762829.732887:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04288 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.732892:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04288 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.732909:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:3.0:1789762829.732911:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762829.732915:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04288 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.732919:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04288 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762829.732930:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000533a3c42 reply state 00000000844e6303 transno 4294967523 ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b0428f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762829.732936:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762829.732978:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:14210:x1876702015015168:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 1608us (1702us total) trans 4294967523 rc 0/0 00000100:00100000:3.0:1789762829.732984:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 269 00010000:00010000:3.0:1789762829.733527:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b0428f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.733534:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b0428f lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.733539:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b0428f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.733542:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063b200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762829.733545:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b0428f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.733552:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b0428f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.733579:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272ca00) 00010000:00010000:3.0:1789762829.733583:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762829.733586:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b0428f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.733593:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b0428f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.737733:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762829.737779:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015015296 00000100:00100000:3.0:1789762829.737785:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015015296 00000100:00100000:3.0:1789762829.737906:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 270 00000100:00100000:3.0:1789762829.737915:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc557800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14211:x1876702015015296:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762829.737929:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762829.737941:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04296 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd5fa expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.737978:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b000/0x416193e012b0429d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.737986:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b000/0x416193e012b0429d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.737994:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b000/0x416193e012b0429d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762829.738011:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762829.738014:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762829.738026:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b000/0x416193e012b0429d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.738053:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b04296 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5fa expref: 8 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.738068:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b000/0x416193e012b0429d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5fa expref: 7 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.738076:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:3.0:1789762829.738079:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762829.738083:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e9063b000, rc 0) 00000100:00100000:3.0:1789762829.738161:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc557800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:14211:x1876702015015296:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 248us (432us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762829.738167:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 270 00000100:00100000:0.0:1789762829.739133:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762829.739183:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015015552 00000100:00100000:1.0:1789762829.739189:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015015552 00000100:00100000:1.0:1789762829.739223:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 271 00000100:00100000:1.0:1789762829.739233:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:14211:x1876702015015552:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762829.739246:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762829.739249:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1789762829.739265:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b000/0x416193e012b0429d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd5fa expref: 7 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739283:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:1.0:1789762829.739293:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762829.739303:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b042a4 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd601 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739381:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b042ab lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739388:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b042ab lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739394:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b042ab lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739402:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b042b2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739407:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b042b2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739412:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b042b2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739484:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b042b9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739497:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b042b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739504:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b042b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762829.739515:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 16384 00010000:00010000:1.0:1789762829.739527:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b042b9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739533:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b042b9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739541:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b042b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739548:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762829.739552:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b042b9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739558:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b042b9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739567:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ec2179800) 00010000:00010000:1.0:1789762829.739570:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762829.739574:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b042b9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739589:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b042b9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739605:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b042ab lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739610:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b042ab lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739615:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b042ab lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739618:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762829.739621:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b042ab lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739625:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b042ab lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739630:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179d00) 00010000:00010000:1.0:1789762829.739631:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762829.739633:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b042ab lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739638:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b042ab lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739644:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b042b2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739648:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b042b2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739652:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b042b2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739656:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762829.739659:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b042b2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739663:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b042b2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739669:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179f00) 00010000:00010000:1.0:1789762829.739671:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762829.739672:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b042b2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739676:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b042b2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739695:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b042a4 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd601 expref: 7 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739701:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179d00) 00010000:00010000:1.0:1789762829.739703:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762829.739705:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b042a4 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd601 expref: 7 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.739713:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81483400, rc 0) 00000100:00100000:1.0:1789762829.739742:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14211:x1876702015015552:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 513us (615us total) trans 0 rc 301/301 00000100:00100000:1.0:1789762829.739748:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 271 00000100:00100000:1.0:1789762829.740485:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762829.740509:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015015680 00000100:00100000:3.0:1789762829.740515:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015015680 00000100:00100000:3.0:1789762829.740547:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 272 00000100:00100000:3.0:1789762829.740556:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14211:x1876702015015680:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762829.740564:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762829.740574:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b042c0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd608 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.740602:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063a600/0x416193e012b042c7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.740610:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063a600/0x416193e012b042c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.740616:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063a600/0x416193e012b042c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762829.740637:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762829.740639:0:5805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762829.740651:0:5805:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063a600/0x416193e012b042c7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.740671:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b042c0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd608 expref: 8 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.740686:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063a600/0x416193e012b042c7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd608 expref: 7 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.740695:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:3.0:1789762829.740699:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762829.740702:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e9063a600, rc 0) 00000100:00100000:3.0:1789762829.740741:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:14211:x1876702015015680:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 186us (255us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762829.740750:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 272 00000100:00100000:1.0:1789762829.741589:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762829.741616:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015015936 00000100:00100000:3.0:1789762829.741622:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015015936 00000100:00100000:3.0:1789762829.741653:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 273 00000100:00100000:3.0:1789762829.741663:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:14211:x1876702015015936:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762829.741673:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762829.741683:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b042ce lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd60f expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741716:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042d5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741740:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042d5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741749:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042d5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741765:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042dc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741771:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042dc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741776:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042dc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762829.741817:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 16384 00010000:00010000:3.0:1789762829.741882:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042d5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741888:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042d5 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741893:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042d5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741896:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063be00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762829.741899:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042d5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741903:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042d5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741920:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:3.0:1789762829.741921:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762829.741924:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042d5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741928:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042d5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741935:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042dc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741939:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042dc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741943:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042dc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741948:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063b600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762829.741950:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042dc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741954:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042dc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741958:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272cb00) 00010000:00010000:3.0:1789762829.741960:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762829.741961:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042dc lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741965:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042dc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.741997:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b042ce lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd60f expref: 9 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.742004:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:3.0:1789762829.742006:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762829.742008:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b042ce lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd60f expref: 9 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.742018:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e9063aa00, rc 0) 00000100:00100000:3.0:1789762829.742047:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:14211:x1876702015015936:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 388us (461us total) trans 4294967524 rc 301/301 00000100:00100000:3.0:1789762829.742056:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 273 00000100:00100000:2.0:1789762829.742933:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762829.742992:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015016192 00000100:00100000:1.0:1789762829.742998:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015016192 00000100:00100000:1.0:1789762829.743033:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 274 00000100:00100000:1.0:1789762829.743043:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81396a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:14211:x1876702015016192:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:1.0:1789762829.743078:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b042ea lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.743185:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b042ea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.743194:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b042ea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762829.743359:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e0652e97 reply state 00000000f3f6ab63 transno 4294967525 ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b042ea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.743367:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:1.0:1789762829.743372:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 16384 00000100:00100000:1.0:1789762829.743418:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81396a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14211:x1876702015016192:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 377us (488us total) trans 4294967525 rc 0/0 00000100:00100000:1.0:1789762829.743424:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 274 00010000:00010000:2.0:1789762829.744129:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b042ea lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.744153:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b042ea lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.744160:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b042ea lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.744165:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762829.744169:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b042ea lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.744174:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b042ea lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762829.744184:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ec2179500) 00010000:00010000:2.0:1789762829.744186:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789762829.744189:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b042ea lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762829.744206:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:2.0:1789762829.744220:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b042ea lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.744252:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015016448 00000100:00100000:1.0:1789762829.744258:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015016448 00000100:00100000:1.0:1789762829.744297:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 84 00000100:00100000:1.0:1789762829.744309:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:14211:x1876702015016448:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:1.0:1789762829.744422:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902de300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14211:x1876702015016448:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 116us (221us total) trans 4294967526 rc 0/0 00000100:00100000:1.0:1789762829.744428:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 84 00000100:00100000:1.0:1789762829.747854:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762829.747888:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015016576 00000100:00100000:3.0:1789762829.747894:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015016576 00000100:00100000:3.0:1789762829.747928:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 275 00000100:00100000:3.0:1789762829.747936:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc557100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14212:x1876702015016576:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:3.0:1789762829.747948:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762829.747961:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b042f1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd616 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748001:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042f8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748011:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042f8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748019:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042f8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748036:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042ff lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748041:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748045:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748216:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04306 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748223:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04306 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748229:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04306 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762829.748246:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 16384 00000004:00200000:3.0:1789762829.748249:0:5805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762829.748260:0:5805:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04306 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748267:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042f8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748272:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042f8 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748281:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042f8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748294:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063b600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762829.748297:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042f8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748302:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042f8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748311:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:3.0:1789762829.748313:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762829.748316:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042f8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748321:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b600/0x416193e012b042f8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748329:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042ff lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748333:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042ff lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748338:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042ff lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748342:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063be00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762829.748344:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042ff lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748348:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042ff lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748362:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272cb00) 00010000:00010000:3.0:1789762829.748364:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762829.748367:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042ff lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748372:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b042ff lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748391:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b042f1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd616 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748400:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04306 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd616 expref: 11 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.748405:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981e9272c000) 00010000:00010000:3.0:1789762829.748407:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762829.748409:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e9063b200, rc 0) 00000100:00100000:3.0:1789762829.748441:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc557100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14212:x1876702015016576:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 507us (589us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762829.748448:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 275 00000100:00100000:0.0:1789762829.749374:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762829.749398:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015016832 00000100:00100000:3.0:1789762829.749403:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015016832 00000100:00100000:3.0:1789762829.749429:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 276 00000100:00100000:3.0:1789762829.749435:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14212:x1876702015016832:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:3.0:1789762829.749457:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762829.749468:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04306 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd616 expref: 11 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.749486:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981e9272c000) 00010000:00010000:3.0:1789762829.749491:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762829.749502:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b0430d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.749508:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b0430d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.749513:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b0430d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762829.749655:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f8195a8d reply state 000000008bac749c transno 4294967527 ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b0430d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762829.749666:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762829.749672:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 16384 00000100:00100000:3.0:1789762829.749717:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902df100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14212:x1876702015016832:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 281us (343us total) trans 4294967527 rc 0/0 00000100:00100000:3.0:1789762829.749725:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 276 00010000:00010000:3.0:1789762829.750228:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b0430d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.750235:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b0430d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.750240:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b0430d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.750243:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063ac00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762829.750245:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b0430d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.750256:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b0430d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.750264:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981e9272c800) 00010000:00010000:3.0:1789762829.750265:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789762829.750268:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b0430d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.750271:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b0430d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762829.763780:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762829.763813:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015016960 00000100:00100000:1.0:1789762829.763821:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015016960 00000100:00100000:1.0:1789762829.763857:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 277 00000100:00100000:1.0:1789762829.763866:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14216:x1876702015016960:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:1.0:1789762829.763881:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762829.763892:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b04314 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd61d expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.763929:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b0431b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.763938:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b0431b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.763946:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b0431b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.763955:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04322 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.763960:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04322 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.763965:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04322 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764030:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b04329 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764035:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b04329 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764040:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b04329 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762829.764063:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 16384 00000004:00200000:1.0:1789762829.764067:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762829.764112:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b04329 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764121:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b0431b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764127:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b0431b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764132:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b0431b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764148:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762829.764152:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b0431b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764157:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b0431b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764166:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:1.0:1789762829.764169:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762829.764171:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b0431b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764176:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b0431b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764189:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04322 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764201:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04322 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764208:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04322 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764215:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762829.764219:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04322 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764227:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04322 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764235:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179d00) 00010000:00010000:1.0:1789762829.764237:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762829.764239:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04322 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764244:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483a00/0x416193e012b04322 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764272:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b04314 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd61d expref: 13 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764287:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b04329 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd61d expref: 12 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.764296:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:1.0:1789762829.764299:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789762829.764302:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81482800, rc 0) 00000100:00100000:1.0:1789762829.764345:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14216:x1876702015016960:12345-192.168.203.48@tcp:101:dd.500 Request processed in 480us (567us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762829.764351:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 277 00000100:00100000:0.0:1789762829.765199:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762829.765229:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015017216 00000100:00100000:3.0:1789762829.765235:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015017216 00000100:00100000:3.0:1789762829.765264:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 278 00000100:00100000:3.0:1789762829.765270:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14216:x1876702015017216:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762829.765278:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762829.765286:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b04330 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd624 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.765344:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04337 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.765351:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04337 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.765357:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04337 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.765387:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b0433e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.765398:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b0433e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.765404:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b0433e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762829.765599:0:6868:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc555180 pname:cluuid:pid:xid:nid:opc:job osp-pre-0-0:lustre-MDT0000-mdtlov_UUID:6868:1876702022171520:0@lo:5:osp-pre-0-0.0 00000100:00100000:1.0:1789762829.765645:0:6868:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762829.765683:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022171520 00000100:00100000:0.0:1789762829.765688:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022171520 00000100:00100000:1.0:1789762829.765706:0:6868:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ea29d7780 going to sleep for 11 seconds 00000100:00100000:0.0:1789762829.765723:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:0.0:1789762829.765734:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:6868:x1876702022171520:12345-0@lo:5:osp-pre-0-0.0 00002000:00080000:0.0:1789762829.765765:0:10916:0:(ofd_dev.c:1747:ofd_create_hdl()) lustre-OST0000: reserve 32 objects in group 0x240000400 at 34 00000004:00080000:3.0:1789762829.765906:0:5805:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x12:0x0], index 0: 0 00000004:00080000:3.0:1789762829.765915:0:5805:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xc:0x0], index 1: 0 00010000:00010000:3.0:1789762829.766057:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b04345 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.766068:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b04345 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.766077:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b04345 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.766233:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04337 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.766240:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04337 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.766245:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04337 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.766249:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063b200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762829.766253:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04337 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.766258:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04337 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.766273:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:3.0:1789762829.766276:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762829.766279:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04337 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.766284:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04337 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762829.766294:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004519b269 reply state 00000000cb6d65c1 transno 4294967528 ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b0433e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762829.766300:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789762829.766320:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b04330 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd624 expref: 15 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.766331:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b04345 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd624 expref: 14 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.766338:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2e:0x0].0x0 (ffff981e9272c000) 00010000:00010000:3.0:1789762829.766340:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762829.766342:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e9063be00, rc 0) 00000100:00100000:3.0:1789762829.766383:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902ddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:14216:x1876702015017216:12345-192.168.203.48@tcp:101:dd.500 Request processed in 1112us (1184us total) trans 4294967528 rc 0/0 00000100:00100000:3.0:1789762829.766391:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 278 00010000:00010000:1.0:1789762829.766922:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b0433e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.766930:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b0433e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.766936:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b0433e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.766940:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063aa00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762829.766942:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b0433e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.766949:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b0433e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.766958:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x77289514 (ffff981e9272c800) 00010000:00010000:1.0:1789762829.766961:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762829.766964:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b0433e lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.766968:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063aa00/0x416193e012b0433e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762829.768823:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762829.768892:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015017344 00000100:00100000:3.0:1789762829.768897:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015017344 00000100:00100000:3.0:1789762829.768934:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 172 00000100:00100000:3.0:1789762829.768944:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:14216:x1876702015017344:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762829.768958:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762829.768973:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003821af31/0x416193e012b04353 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd62b expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762829.768989:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x12:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762829.769001:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=353s v=(0 0) 00010000:00010000:3.0:1789762829.769005:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762829.769012:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003821af31/0x416193e012b04353 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd62b expref: 6 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.769024:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003821af31/0x416193e012b04353 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd62b expref: 6 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762829.769032:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e9063b600, rc 0) 00000100:00100000:3.0:1789762829.769073:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14216:x1876702015017344:12345-192.168.203.48@tcp:101:dd.500 Request processed in 130us (252us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762829.769082:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 172 00000100:00100000:3.0:1789762829.776259:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762829.776298:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015017472 00000100:00100000:2.0:1789762829.776306:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015017472 00002000:00100000:2.0:1789762829.776348:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x12:0x0] (0->1048575) req@ffff981ebad81f80 x1876702015017472/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/0 e 0 to 0 dl 1789762840 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762829.776357:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad81f80 with x1876702015017472 ext(0->1048575) 00002000:00010000:2.0:1789762829.776362:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad81f80 00000100:00100000:2.0:1789762829.776376:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328695039000 time 328695124039 state 0x0 ref 1 00000100:00100000:2.0:1789762829.776383:0:14913:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 8: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 328695039000 time 328695124039 state 0x0 ref 1 00000100:00100000:2.0:1789762829.776396:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 3 deadline 353180303743 time 353080303743 state 0x1 ref 1 00000100:00100000:2.0:1789762829.776420:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 353180303743 time 353080329490 state 0x0 ref 1 00000100:00100000:2.0:1789762829.776430:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015017472:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762829.780684:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:6868:x1876702022171520:12345-0@lo:5:osp-pre-0-0.0 Request processed in 14952us (15037us total) trans 4294967912 rc 0/0 00000100:00100000:0.0:1789762829.780691:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:1.0:1789762829.780746:0:6868:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc555180 pname:cluuid:pid:xid:nid:opc:job osp-pre-0-0:lustre-MDT0000-mdtlov_UUID:6868:1876702022171520:0@lo:5:osp-pre-0-0.0 00000004:00080000:1.0:1789762829.780759:0:6868:0:(osp_precreate.c:705:osp_precreate_send()) lustre-OST0000-osc-MDT0000: current precreated pool: [0x240000400:0x12:0x0]-[0x240000400:0x41:0x0] 00000100:00100000:3.0:1789762829.782675:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762829.782723:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015017600 00000100:00100000:1.0:1789762829.782729:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015017600 00002000:00100000:1.0:1789762829.782755:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x12:0x0] (1048576->2097151) req@ffff981ebfdcb100 x1876702015017600/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/0 e 0 to 0 dl 1789762840 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762829.782763:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdcb100 with x1876702015017600 ext(1048576->2097151) 00002000:00010000:1.0:1789762829.782767:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdcb100 00000100:00100000:1.0:1789762829.782779:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 353180329490 time 353080329490 state 0x1 ref 2 00000100:00100000:1.0:1789762829.782800:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 353180329490 time 353086709988 state 0x0 ref 2 00000100:00100000:1.0:1789762829.782812:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702015017600:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762829.783262:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702015017472:12345-192.168.203.48@tcp:4:dd.500 Request processed in 6832us (7006us total) trans 4294967913 rc 0/0 00000100:00100000:0.0:1789762829.783270:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 693 00002000:00100000:0.0:1789762829.783277:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x12:0x0] (0->1048575) req@ffff981ebad81f80 x1876702015017472/t4294967913(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/448 e 0 to 0 dl 1789762840 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762829.783284:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebad81f80 with x1876702015017472 ext(0->1048575) 00002000:00010000:0.0:1789762829.783287:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebad81f80 00000100:00100000:1.0:1789762829.785380:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702015017600:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2569us (2706us total) trans 4294967914 rc 0/0 00000100:00100000:1.0:1789762829.785388:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 694 00000100:00100000:1.0:1789762829.785395:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 353180329490 time 353086709988 state 0x2 ref 0 00002000:00100000:1.0:1789762829.785405:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x12:0x0] (1048576->2097151) req@ffff981ebfdcb100 x1876702015017600/t4294967914(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/448 e 0 to 0 dl 1789762840 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762829.785412:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdcb100 with x1876702015017600 ext(1048576->2097151) 00002000:00010000:1.0:1789762829.785415:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdcb100 00000100:00100000:2.0:1789762829.789581:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762829.789646:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015017728 00000100:00100000:0.0:1789762829.789652:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015017728 00002000:00100000:0.0:1789762829.789673:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x12:0x0] (2097152->3145727) req@ffff981d81395880 x1876702015017728/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/0 e 0 to 0 dl 1789762840 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762829.789686:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395880 with x1876702015017728 ext(2097152->3145727) 00002000:00010000:0.0:1789762829.789689:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81395880 00000100:00100000:0.0:1789762829.789698:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 353180329490 time 353086709988 state 0x0 ref 1 00000100:00100000:0.0:1789762829.789706:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 353186709988 time 353086709988 state 0x1 ref 1 00000100:00100000:0.0:1789762829.789730:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353186709988 time 353093639924 state 0x0 ref 1 00000100:00100000:0.0:1789762829.789738:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702015017728:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762829.792040:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2364:x1876702015017728:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2300us (2458us total) trans 4294967915 rc 0/0 00000100:00100000:0.0:1789762829.792047:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 695 00000100:00100000:0.0:1789762829.792053:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353186709988 time 353093639924 state 0x2 ref 0 00002000:00100000:0.0:1789762829.792062:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x12:0x0] (2097152->3145727) req@ffff981d81395880 x1876702015017728/t4294967915(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/448 e 0 to 0 dl 1789762840 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762829.792070:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395880 with x1876702015017728 ext(2097152->3145727) 00002000:00010000:0.0:1789762829.792073:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81395880 00000100:00100000:2.0:1789762829.797403:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762829.797444:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015017856 00000100:00100000:3.0:1789762829.797450:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015017856 00002000:00100000:3.0:1789762829.797471:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x12:0x0] (3145728->4194303) req@ffff981d81395f80 x1876702015017856/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/0 e 0 to 0 dl 1789762840 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762829.797479:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395f80 with x1876702015017856 ext(3145728->4194303) 00002000:00010000:3.0:1789762829.797482:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81395f80 00000100:00100000:3.0:1789762829.797490:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353186709988 time 353093639924 state 0x0 ref 1 00000100:00100000:3.0:1789762829.797504:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353193639924 time 353093639924 state 0x1 ref 1 00000100:00100000:2.0:1789762829.797641:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762829.797708:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015017984 00000100:00100000:0.0:1789762829.797714:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015017984 00000100:00100000:0.0:1789762829.797741:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 173 00000100:00100000:0.0:1789762829.797746:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0+-99:2362:x1876702015017984:12345-192.168.203.48@tcp:8:dd.500 00010000:00080000:0.0:1789762829.797773:0:6648:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp t0 exp 0000000000000000 cur 353 last 0 00000020:00000080:0.0:1789762829.797798:0:6648:0:(genops.c:1385:class_connect()) connect: client f1bd28de-ee78-4d18-bad3-fa7cec968893, cookie 0x416193e012b0435a 00002000:00100000:0.0:1789762829.797806:0:6648:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli f1bd28de-ee78-4d18-bad3-fa7cec968893/ffff981ebfa2e800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:0.0:1789762829.797813:0:6648:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:0.0:1789762829.797925:0:6648:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:0.0:1789762829.797948:0:6648:0:(import.c:65:import_set_state_nolock()) ffff981ebfa2f800 : changing import state from RECOVER to FULL 00000100:00100000:0.0:1789762829.797981:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2362:x1876702015017984:12345-192.168.203.48@tcp:8:dd.500 Request processed in 234us (340us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762829.797987:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 173 00000100:00100000:2.0:1789762829.798711:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762829.798761:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015018112 00000100:00100000:1.0:1789762829.798767:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015018112 00000100:00100000:1.0:1789762829.798807:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 174 00000100:00080000:2.0:1789762829.798811:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762829.798815:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81397800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14216:x1876702015018112:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:1.0:1789762829.798824:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762829.798842:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b04361 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd632 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762829.798860:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xc:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762829.798874:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=353s v=(0 0) 00010000:00010000:1.0:1789762829.798878:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762829.798885:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b04361 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd632 expref: 7 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.798899:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b04361 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd632 expref: 7 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762829.798908:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81483400, rc 0) 00000100:00100000:1.0:1789762829.798931:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81397800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:14216:x1876702015018112:12345-192.168.203.48@tcp:101:dd.500 Request processed in 118us (221us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762829.798936:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 174 00000100:00100000:2.0:1789762829.806463:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762829.806512:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015018240 00000100:00100000:1.0:1789762829.806518:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015018240 00002000:00100000:1.0:1789762829.806542:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0xc:0x0] (0->1048575) req@ffff981d81397100 x1876702015018240/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/0 e 0 to 0 dl 1789762840 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762829.806553:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81397100 with x1876702015018240 ext(0->1048575) 00002000:00010000:1.0:1789762829.806558:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81397100 00000100:00100000:1.0:1789762829.806573:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353193639924 time 353093639924 state 0x1 ref 2 00000100:00100000:2.0:1789762829.813161:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762829.813206:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015018368 00000100:00100000:1.0:1789762829.813211:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015018368 00002000:00100000:1.0:1789762829.813231:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xc:0x0] (1048576->2097151) req@ffff981d81394e00 x1876702015018368/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/0 e 0 to 0 dl 1789762840 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762829.813238:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81394e00 with x1876702015018368 ext(1048576->2097151) 00002000:00010000:1.0:1789762829.813241:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81394e00 00000100:00100000:1.0:1789762829.813250:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353193639924 time 353093639924 state 0x1 ref 3 00000100:00100000:1.0:1789762829.889834:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353293728677 time 353193728677 state 0x1 ref 3 00000100:00100000:1.0:1789762829.889860:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702015017856:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762829.892525:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:2363:x1876702015017856:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2675us (95122us total) trans 4294967916 rc 0/0 00000100:00100000:1.0:1789762829.892533:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 696 00002000:00100000:1.0:1789762829.892542:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x12:0x0] (3145728->4194303) req@ffff981d81395f80 x1876702015017856/t4294967916(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/448 e 0 to 0 dl 1789762840 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762829.892551:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395f80 with x1876702015017856 ext(3145728->4194303) 00002000:00010000:1.0:1789762829.892557:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81395f80 00000100:00100000:3.0:1789762829.989958:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353393839516 time 353293839516 state 0x1 ref 2 00000100:00100000:3.0:1789762829.989990:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81397100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702015018240:12345-192.168.203.48@tcp:4:dd.500 00000001:00100000:1.0:1789762830.006947:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000001:00080000:1.0:1789762830.020220:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967399 is committed 00000100:00100000:0.0:1789762830.020286:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81397100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2363:x1876702015018240:12345-192.168.203.48@tcp:4:dd.500 Request processed in 30303us (213817us total) trans 4294967399 rc 0/0 00000100:00100000:0.0:1789762830.020301:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 697 00002000:00100000:0.0:1789762830.020315:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0xc:0x0] (0->1048575) req@ffff981d81397100 x1876702015018240/t4294967399(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/448 e 0 to 0 dl 1789762840 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762830.020324:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81397100 with x1876702015018240 ext(0->1048575) 00002000:00010000:0.0:1789762830.020329:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81397100 00000100:00100000:3.0:1789762830.022658:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762830.022709:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015018496 00000100:00100000:0.0:1789762830.022717:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015018496 00002000:00100000:0.0:1789762830.022747:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xc:0x0] (2097152->3145727) req@ffff981ebfdc8a80 x1876702015018496/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762830.022755:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc8a80 with x1876702015018496 ext(2097152->3145727) 00002000:00010000:0.0:1789762830.022758:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdc8a80 00000100:00100000:0.0:1789762830.022775:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353393839516 time 353293839516 state 0x1 ref 2 00000100:00100000:2.0:1789762830.029295:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762830.029369:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015018624 00000100:00100000:3.0:1789762830.029380:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015018624 00002000:00100000:3.0:1789762830.029422:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xc:0x0] (3145728->4194303) req@ffff981d81396680 x1876702015018624/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762830.029442:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81396680 with x1876702015018624 ext(3145728->4194303) 00002000:00010000:3.0:1789762830.029448:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81396680 00000100:00100000:3.0:1789762830.029470:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353393839516 time 353293839516 state 0x1 ref 3 00000100:00100000:2.0:1789762830.035728:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762830.035779:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015018752 00000100:00100000:3.0:1789762830.035785:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015018752 00002000:00100000:3.0:1789762830.035823:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x12:0x0] (4194304->5242879) req@ffff981d81395180 x1876702015018752/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00000100:00080000:2.0:1789762830.035833:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00100000:3.0:1789762830.035839:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395180 with x1876702015018752 ext(4194304->5242879) 00002000:00010000:3.0:1789762830.035847:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81395180 00000100:00100000:3.0:1789762830.035862:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353393839516 time 353293839516 state 0x1 ref 4 00000100:00100000:3.0:1789762830.042487:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762830.042578:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015018880 00000100:00100000:0.0:1789762830.042589:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015018880 00002000:00100000:0.0:1789762830.042627:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x12:0x0] (5242880->6291455) req@ffff981ebfdc9c00 x1876702015018880/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762830.042638:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc9c00 with x1876702015018880 ext(5242880->6291455) 00002000:00010000:0.0:1789762830.042643:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdc9c00 00000100:00100000:0.0:1789762830.042659:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353393839516 time 353293839516 state 0x1 ref 5 00000100:00100000:2.0:1789762830.049195:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762830.049271:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015019008 00000100:00100000:0.0:1789762830.049277:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015019008 00002000:00100000:0.0:1789762830.049296:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x240000400:0x12:0x0] (6291456->7340031) req@ffff981ebfdfb100 x1876702015019008/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762830.049304:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfb100 with x1876702015019008 ext(6291456->7340031) 00002000:00010000:0.0:1789762830.049307:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdfb100 00000100:00100000:0.0:1789762830.049316:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353393839516 time 353293839516 state 0x1 ref 6 00000100:00100000:2.0:1789762830.055550:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.055637:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015019136 00000100:00100000:1.0:1789762830.055645:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015019136 00002000:00100000:1.0:1789762830.055673:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x12:0x0] (7340032->8388607) req@ffff981ebac58a80 x1876702015019136/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762830.055682:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac58a80 with x1876702015019136 ext(7340032->8388607) 00002000:00010000:1.0:1789762830.055686:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac58a80 00000100:00100000:1.0:1789762830.055695:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353393839516 time 353293839516 state 0x1 ref 7 00000100:00100000:1.0:1789762830.090022:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353493919474 time 353393919474 state 0x1 ref 7 00000100:00100000:1.0:1789762830.090045:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702015018368:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762830.093125:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702015018368:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3086us (279963us total) trans 4294967400 rc 0/0 00000100:00100000:1.0:1789762830.093132:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 698 00002000:00100000:1.0:1789762830.093154:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0xc:0x0] (1048576->2097151) req@ffff981d81394e00 x1876702015018368/t4294967400(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:610/0 lens 488/448 e 0 to 0 dl 1789762840 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762830.093162:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81394e00 with x1876702015018368 ext(1048576->2097151) 00002000:00010000:1.0:1789762830.093168:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81394e00 00000100:00100000:3.0:1789762830.095027:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.095124:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015019264 00000100:00100000:1.0:1789762830.095132:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015019264 00002000:00100000:1.0:1789762830.095175:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xc:0x0] (4194304->5242879) req@ffff981ebfdb8e00 x1876702015019264/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762830.095184:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb8e00 with x1876702015019264 ext(4194304->5242879) 00002000:00010000:1.0:1789762830.095188:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdb8e00 00000100:00100000:1.0:1789762830.095200:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353493919474 time 353393919474 state 0x1 ref 7 00000100:00100000:2.0:1789762830.101920:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762830.102002:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015019392 00000100:00100000:0.0:1789762830.102008:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015019392 00002000:00100000:0.0:1789762830.102030:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0xc:0x0] (5242880->6291455) req@ffff981ebac58000 x1876702015019392/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762830.102038:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac58000 with x1876702015019392 ext(5242880->6291455) 00002000:00010000:0.0:1789762830.102041:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac58000 00000100:00100000:0.0:1789762830.102050:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353493919474 time 353393919474 state 0x1 ref 8 00000100:00100000:0.0:1789762830.190167:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353594062267 time 353494062267 state 0x1 ref 8 00000100:00100000:0.0:1789762830.190195:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:2363:x1876702015018496:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762830.193547:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:2363:x1876702015018496:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3362us (170895us total) trans 4294967401 rc 0/0 00000100:00100000:0.0:1789762830.193554:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 699 00002000:00100000:0.0:1789762830.193569:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0xc:0x0] (2097152->3145727) req@ffff981ebfdc8a80 x1876702015018496/t4294967401(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762830.193577:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc8a80 with x1876702015018496 ext(2097152->3145727) 00002000:00010000:0.0:1789762830.193583:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdc8a80 00000100:00100000:3.0:1789762830.195729:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762830.195806:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015019520 00000100:00100000:2.0:1789762830.195814:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015019520 00002000:00100000:2.0:1789762830.195842:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xc:0x0] (6291456->7340031) req@ffff981ebfdba680 x1876702015019520/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762830.195850:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdba680 with x1876702015019520 ext(6291456->7340031) 00002000:00010000:2.0:1789762830.195853:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdba680 00000100:00100000:2.0:1789762830.195866:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353594062267 time 353494062267 state 0x1 ref 8 00000100:00100000:2.0:1789762830.203326:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762830.203357:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762830.203395:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015019648 00000100:00100000:3.0:1789762830.203405:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015019648 00000100:00100000:0.0:1789762830.203444:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015019904 00000100:00100000:0.0:1789762830.203451:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015019904 00002000:00100000:3.0:1789762830.203453:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xc:0x0] (7340032->8388607) req@ffff981ebac59500 x1876702015019648/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762830.203464:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac59500 with x1876702015019648 ext(7340032->8388607) 00002000:00010000:3.0:1789762830.203469:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac59500 00000100:00100000:3.0:1789762830.203489:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353594062267 time 353494062267 state 0x1 ref 9 00000100:00100000:0.0:1789762830.203512:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 85 00000100:00080000:2.0:1789762830.203519:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762830.203526:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14216:x1876702015019904:12345-192.168.203.48@tcp:35:dd.500 00000001:00080000:0.0:1789762830.203698:0:11718:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967529 00000100:00100000:0.0:1789762830.203766:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:14216:x1876702015019904:12345-192.168.203.48@tcp:35:dd.500 Request processed in 246us (408us total) trans 4294967530 rc 0/0 00000100:00100000:0.0:1789762830.203772:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 85 00000020:00080000:0.0:1789762830.203775:0:11718:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1789762830.236843:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762830.236908:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015020160 00000100:00100000:2.0:1789762830.236921:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015020160 00000100:00100000:2.0:1789762830.236975:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 279 00000100:00100000:2.0:1789762830.236983:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:14227:x1876702015020160:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762830.237003:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762830.237020:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a000/0x416193e012b04368 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd639 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237160:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b0436f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237190:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b0436f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237202:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b0436f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237219:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b04376 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237225:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b04376 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237230:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b04376 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789762830.237621:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xd:0x0], index 1: 0 00000004:00080000:2.0:1789762830.237627:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x13:0x0], index 0: 0 00010000:00010000:2.0:1789762830.237751:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a200/0x416193e012b0437d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237759:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a200/0x416193e012b0437d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237764:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a200/0x416193e012b0437d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237793:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b0436f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237798:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b0436f lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237812:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b0436f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237819:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1be00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762830.237824:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b0436f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237832:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b0436f lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237845:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:2.0:1789762830.237849:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762830.237854:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b0436f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237862:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b0436f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762830.237874:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004f3b5f8e reply state 0000000056ccbd7b transno 4294967531 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b04376 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762830.237881:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789762830.237902:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a000/0x416193e012b04368 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd639 expref: 19 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237912:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a200/0x416193e012b0437d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd639 expref: 18 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762830.237917:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff981e82a26600) 00010000:00010000:2.0:1789762830.237924:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762830.237926:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87c1a200, rc 0) 00000100:00100000:2.0:1789762830.237985:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:14227:x1876702015020160:12345-192.168.203.48@tcp:101:dd.0 Request processed in 1003us (1157us total) trans 4294967531 rc 0/0 00000100:00100000:2.0:1789762830.237990:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 279 00010000:00010000:3.0:1789762830.238642:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b04376 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.238656:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b04376 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.238666:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b04376 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.238673:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ae00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762830.238679:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b04376 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.238687:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b04376 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.238705:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x992ad74f (ffff981e82a26900) 00010000:00010000:3.0:1789762830.238709:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762830.238726:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b04376 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762830.238733:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b04376 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762830.244068:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762830.244170:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015020288 00000100:00100000:0.0:1789762830.244177:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015020288 00000100:00100000:3.0:1789762830.244204:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762830.244237:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015020416 00000100:00100000:0.0:1789762830.244241:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015020416 00000100:00100000:1.0:1789762830.244264:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 175 00000100:00100000:0.0:1789762830.244274:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 176 00000100:00100000:1.0:1789762830.244282:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:14228:x1876702015020288:12345-192.168.203.48@tcp:101:dd.0 00000100:00100000:0.0:1789762830.244290:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:14232:x1876702015020416:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762830.244295:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762830.244308:0:10916:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762830.244309:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003cb287b8/0x416193e012b0438b lrc: 2/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd640 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.244321:0:10916:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000481d0182/0x416193e012b04392 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd647 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762830.244331:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xd:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00002000:00010000:0.0:1789762830.244340:0:10916:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x13:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762830.244346:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=353s v=(0 0) 00010000:00010000:1.0:1789762830.244351:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762830.244354:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=353s v=(0 0) 00010000:00010000:0.0:1789762830.244360:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762830.244361:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003cb287b8/0x416193e012b0438b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd640 expref: 16 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.244368:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000481d0182/0x416193e012b04392 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd647 expref: 19 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.244377:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003cb287b8/0x416193e012b0438b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd640 expref: 16 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762830.244385:0:10916:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000481d0182/0x416193e012b04392 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd647 expref: 19 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762830.244389:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81483a00, rc 0) 00010000:00010000:0.0:1789762830.244397:0:10916:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbe400, rc 0) 00000100:00100000:1.0:1789762830.244430:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:14228:x1876702015020288:12345-192.168.203.48@tcp:101:dd.0 Request processed in 155us (364us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762830.244431:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:14232:x1876702015020416:12345-192.168.203.48@tcp:101:dd.0 Request processed in 147us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762830.244446:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 175 00000100:00100000:0.0:1789762830.244455:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 176 00010000:00010000:0.0:1789762830.263191:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762830.290297:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353694189091 time 353594189091 state 0x1 ref 9 00000100:00100000:1.0:1789762830.290324:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81396680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:2364:x1876702015018624:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762830.293986:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81396680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:2364:x1876702015018624:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3673us (264693us total) trans 4294967402 rc 0/0 00000100:00100000:1.0:1789762830.293998:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 700 00002000:00100000:1.0:1789762830.294007:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xc:0x0] (3145728->4194303) req@ffff981d81396680 x1876702015018624/t4294967402(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762830.294015:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81396680 with x1876702015018624 ext(3145728->4194303) 00002000:00010000:1.0:1789762830.294021:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81396680 00000100:00100000:3.0:1789762830.297633:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762830.297685:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015020544 00000100:00100000:2.0:1789762830.297693:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015020544 00002000:00100000:2.0:1789762830.297726:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xd:0x0] (2097152->3145727) req@ffff981ebfdb8a80 x1876702015020544/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762830.297734:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb8a80 with x1876702015020544 ext(2097152->3145727) 00002000:00010000:2.0:1789762830.297737:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdb8a80 00000100:00100000:2.0:1789762830.297748:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 323050532720 time 323050446761 state 0x0 ref 1 00000100:00100000:2.0:1789762830.297755:0:11723:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 8: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 323050532720 time 323050446761 state 0x0 ref 1 00000100:00100000:2.0:1789762830.297772:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 3 deadline 353651676441 time 353601676441 state 0x1 ref 1 00000100:00100000:2.0:1789762830.300956:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.301004:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015020672 00000100:00100000:1.0:1789762830.301010:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015020672 00002000:00100000:1.0:1789762830.301032:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0xd:0x0] (0->1048575) req@ffff981ebac5b480 x1876702015020672/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.301039:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac5b480 with x1876702015020672 ext(0->1048575) 00002000:00010000:1.0:1789762830.301042:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac5b480 00000100:00100000:1.0:1789762830.301051:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 3 deadline 353651676441 time 353601676441 state 0x1 ref 2 00000100:00100000:2.0:1789762830.307913:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762830.307993:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015020800 00000100:00100000:0.0:1789762830.307999:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015020800 00002000:00100000:0.0:1789762830.308029:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x13:0x0] (2097152->3145727) req@ffff981ebac59880 x1876702015020800/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762830.308040:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac59880 with x1876702015020800 ext(2097152->3145727) 00002000:00010000:0.0:1789762830.308045:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac59880 00000100:00100000:0.0:1789762830.308058:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 3 deadline 353651676441 time 353601676441 state 0x1 ref 3 00000100:00100000:3.0:1789762830.314521:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.315183:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015020928 00000100:00100000:1.0:1789762830.315195:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015020928 00002000:00100000:1.0:1789762830.315247:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x13:0x0] (1048576->2097151) req@ffff981ebfdb8000 x1876702015020928/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.315257:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb8000 with x1876702015020928 ext(1048576->2097151) 00002000:00010000:1.0:1789762830.315263:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdb8000 00000100:00100000:1.0:1789762830.315284:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 3 deadline 353651676441 time 353601676441 state 0x1 ref 4 00000100:00100000:2.0:1789762830.331156:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.331207:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015021056 00000100:00100000:1.0:1789762830.331216:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015021056 00002000:00100000:1.0:1789762830.331255:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x13:0x0] (0->1048575) req@ffff981ebac5b800 x1876702015021056/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.331268:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac5b800 with x1876702015021056 ext(0->1048575) 00002000:00010000:1.0:1789762830.331273:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac5b800 00000100:00100000:1.0:1789762830.331288:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 3 deadline 353651676441 time 353601676441 state 0x1 ref 5 00000100:00100000:2.0:1789762830.335879:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762830.335930:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015021184 00000100:00100000:0.0:1789762830.335936:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015021184 00002000:00100000:0.0:1789762830.335961:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x13:0x0] (3145728->4194303) req@ffff981ebac5aa00 x1876702015021184/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762830.335969:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac5aa00 with x1876702015021184 ext(3145728->4194303) 00002000:00010000:0.0:1789762830.335972:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac5aa00 00000100:00100000:0.0:1789762830.335981:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 3 deadline 353651676441 time 353601676441 state 0x1 ref 6 00000100:00100000:3.0:1789762830.346023:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762830.346075:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015021312 00000100:00100000:0.0:1789762830.346082:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015021312 00002000:00100000:0.0:1789762830.346129:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x240000400:0x13:0x0] (5242880->6291455) req@ffff981ebfdbad80 x1876702015021312/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762830.346150:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbad80 with x1876702015021312 ext(5242880->6291455) 00002000:00010000:0.0:1789762830.346153:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdbad80 00000100:00100000:0.0:1789762830.346163:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 3 deadline 353651676441 time 353601676441 state 0x1 ref 7 00000100:00100000:0.0:1789762830.347847:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 2 deadline 353701752731 time 353651752731 state 0x1 ref 7 00000100:00100000:0.0:1789762830.347860:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:2366:x1876702015020544:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762830.352743:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:2366:x1876702015020544:12345-192.168.203.48@tcp:4:dd.0 Request processed in 4883us (55118us total) trans 4294967403 rc 0/0 00000100:00100000:0.0:1789762830.352751:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 709 00002000:00100000:0.0:1789762830.352757:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xd:0x0] (2097152->3145727) req@ffff981ebfdb8a80 x1876702015020544/t4294967403(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762830.352770:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb8a80 with x1876702015020544 ext(2097152->3145727) 00002000:00010000:0.0:1789762830.352773:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdb8a80 00000100:00100000:3.0:1789762830.357102:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.357153:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015021440 00000100:00100000:1.0:1789762830.357158:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015021440 00002000:00100000:1.0:1789762830.357180:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xd:0x0] (7340032->8388607) req@ffff981ebfdbb800 x1876702015021440/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.357188:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbb800 with x1876702015021440 ext(7340032->8388607) 00002000:00010000:1.0:1789762830.357191:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdbb800 00000100:00100000:1.0:1789762830.357201:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 2 deadline 353701752731 time 353651752731 state 0x1 ref 7 00000100:00100000:2.0:1789762830.357857:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.357893:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015021568 00000100:00100000:1.0:1789762830.357898:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015021568 00002000:00100000:1.0:1789762830.357915:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xd:0x0] (1048576->2097151) req@ffff981ebac5a680 x1876702015021568/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.357922:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac5a680 with x1876702015021568 ext(1048576->2097151) 00002000:00010000:1.0:1789762830.357924:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac5a680 00000100:00100000:1.0:1789762830.357933:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 2 deadline 353701752731 time 353651752731 state 0x1 ref 8 00000100:00100000:0.0:1789762830.390341:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353794235994 time 353694235994 state 0x1 ref 8 00000100:00100000:0.0:1789762830.390361:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2364:x1876702015018752:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762830.392917:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2364:x1876702015018752:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2559us (357189us total) trans 4294967917 rc 0/0 00000100:00100000:0.0:1789762830.392926:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 701 00002000:00100000:0.0:1789762830.392936:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x12:0x0] (4194304->5242879) req@ffff981d81395180 x1876702015018752/t4294967917(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762830.392944:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395180 with x1876702015018752 ext(4194304->5242879) 00002000:00010000:0.0:1789762830.392950:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81395180 00000100:00100000:0.0:1789762830.392972:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 1 deadline 353746880160 time 353696880160 state 0x1 ref 8 00000100:00100000:0.0:1789762830.392984:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:2365:x1876702015020672:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762830.394723:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.394784:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015021696 00000100:00100000:1.0:1789762830.394793:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015021696 00002000:00100000:1.0:1789762830.394828:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x13:0x0] (6291456->7340031) req@ffff981ebac59180 x1876702015021696/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.394847:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac59180 with x1876702015021696 ext(6291456->7340031) 00002000:00010000:1.0:1789762830.394851:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac59180 00000100:00100000:1.0:1789762830.394866:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 1 deadline 353746880160 time 353696880160 state 0x1 ref 9 00000100:00100000:1.0:1789762830.394902:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353748808044 time 353698808044 state 0x1 ref 9 00000100:00100000:1.0:1789762830.394917:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2366:x1876702015020800:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762830.395445:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2365:x1876702015020672:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2461us (94487us total) trans 4294967404 rc 0/0 00000100:00100000:0.0:1789762830.395455:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 710 00002000:00100000:0.0:1789762830.395464:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0xd:0x0] (0->1048575) req@ffff981ebac5b480 x1876702015020672/t4294967404(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762830.395476:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac5b480 with x1876702015020672 ext(0->1048575) 00002000:00010000:0.0:1789762830.395490:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac5b480 00000100:00100000:3.0:1789762830.398789:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.398837:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015021824 00000100:00100000:1.0:1789762830.398843:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015021824 00002000:00100000:1.0:1789762830.398869:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xd:0x0] (4194304->5242879) req@ffff981ebfdb8380 x1876702015021824/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.398879:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb8380 with x1876702015021824 ext(4194304->5242879) 00002000:00010000:1.0:1789762830.398884:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdb8380 00000100:00100000:1.0:1789762830.398896:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353748808044 time 353698808044 state 0x1 ref 9 00000100:00100000:3.0:1789762830.398999:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.399033:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015021952 00000100:00100000:1.0:1789762830.399037:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015021952 00002000:00100000:1.0:1789762830.399053:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xd:0x0] (5242880->6291455) req@ffff981ebfdbb480 x1876702015021952/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.399060:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbb480 with x1876702015021952 ext(5242880->6291455) 00002000:00010000:1.0:1789762830.399064:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdbb480 00000100:00100000:1.0:1789762830.399075:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353748808044 time 353698808044 state 0x1 ref 10 00000100:00100000:1.0:1789762830.402366:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2366:x1876702015020800:12345-192.168.203.48@tcp:4:dd.0 Request processed in 7452us (94453us total) trans 4294967918 rc 0/0 00000100:00100000:1.0:1789762830.402372:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 711 00002000:00100000:1.0:1789762830.402377:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x13:0x0] (2097152->3145727) req@ffff981ebac59880 x1876702015020800/t4294967918(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.402384:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac59880 with x1876702015020800 ext(2097152->3145727) 00002000:00010000:1.0:1789762830.402387:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac59880 00000100:00100000:3.0:1789762830.403815:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.403857:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015022080 00000100:00100000:1.0:1789762830.403862:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015022080 00002000:00100000:1.0:1789762830.403884:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x13:0x0] (7340032->8388607) req@ffff981ebfdb9c00 x1876702015022080/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.403894:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb9c00 with x1876702015022080 ext(7340032->8388607) 00002000:00010000:1.0:1789762830.403898:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdb9c00 00000100:00100000:1.0:1789762830.403910:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353748808044 time 353698808044 state 0x1 ref 10 00000100:00100000:0.0:1789762830.444985:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353798879262 time 353748879262 state 0x1 ref 10 00000100:00100000:0.0:1789762830.445007:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2365:x1876702015020928:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762830.447912:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2365:x1876702015020928:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2908us (133396us total) trans 4294967919 rc 0/0 00000100:00100000:0.0:1789762830.447924:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 712 00002000:00100000:0.0:1789762830.447937:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x240000400:0x13:0x0] (1048576->2097151) req@ffff981ebfdb8000 x1876702015020928/t4294967919(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762830.447948:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb8000 with x1876702015020928 ext(1048576->2097151) 00002000:00010000:0.0:1789762830.447954:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdb8000 00000100:00100000:3.0:1789762830.449582:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762830.449626:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015022208 00000100:00100000:0.0:1789762830.449634:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015022208 00002000:00100000:0.0:1789762830.449660:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x13:0x0] (4194304->5242879) req@ffff981ebfdb9f80 x1876702015022208/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762830.449667:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb9f80 with x1876702015022208 ext(4194304->5242879) 00002000:00010000:0.0:1789762830.449669:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdb9f80 00000100:00100000:0.0:1789762830.449682:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353798879262 time 353748879262 state 0x1 ref 10 00000100:00100000:0.0:1789762830.490423:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353894317226 time 353794317226 state 0x1 ref 7 00000100:00100000:0.0:1789762830.490444:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2364:x1876702015018880:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762830.493252:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2364:x1876702015018880:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2811us (450763us total) trans 4294967920 rc 0/0 00000100:00100000:0.0:1789762830.493271:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 702 00002000:00100000:0.0:1789762830.493283:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x12:0x0] (5242880->6291455) req@ffff981ebfdc9c00 x1876702015018880/t4294967920(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762830.493294:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc9c00 with x1876702015018880 ext(5242880->6291455) 00002000:00010000:0.0:1789762830.493301:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdc9c00 00000100:00100000:3.0:1789762830.495073:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353848976668 time 353798976668 state 0x1 ref 10 00000100:00100000:3.0:1789762830.495103:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:2363:x1876702015021056:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762830.497745:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:2363:x1876702015021056:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2659us (166594us total) trans 4294967921 rc 0/0 00000100:00100000:0.0:1789762830.497753:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 713 00002000:00100000:0.0:1789762830.497762:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x240000400:0x13:0x0] (0->1048575) req@ffff981ebac5b800 x1876702015021056/t4294967921(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762830.497770:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac5b800 with x1876702015021056 ext(0->1048575) 00002000:00010000:0.0:1789762830.497774:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac5b800 00000100:00100000:1.0:1789762830.545214:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353899109007 time 353849109007 state 0x1 ref 9 00000100:00100000:1.0:1789762830.545239:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2366:x1876702015021184:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762830.547813:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2366:x1876702015021184:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2582us (211933us total) trans 4294967922 rc 0/0 00000100:00100000:1.0:1789762830.547824:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 714 00002000:00100000:1.0:1789762830.547834:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x13:0x0] (3145728->4194303) req@ffff981ebac5aa00 x1876702015021184/t4294967922(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.547847:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac5aa00 with x1876702015021184 ext(3145728->4194303) 00002000:00010000:1.0:1789762830.547853:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac5aa00 00000100:00100000:1.0:1789762830.590503:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 353994396519 time 353894396519 state 0x1 ref 6 00000100:00100000:1.0:1789762830.590532:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:2364:x1876702015019008:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762830.593495:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2364:x1876702015019008:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2972us (544296us total) trans 4294967923 rc 0/0 00000100:00100000:1.0:1789762830.593506:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 703 00002000:00100000:1.0:1789762830.593520:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x12:0x0] (6291456->7340031) req@ffff981ebfdfb100 x1876702015019008/t4294967923(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762830.593531:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfb100 with x1876702015019008 ext(6291456->7340031) 00002000:00010000:1.0:1789762830.593537:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdfb100 00000100:00100000:1.0:1789762830.595243:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353949146794 time 353899146794 state 0x1 ref 8 00000100:00100000:1.0:1789762830.595255:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2364:x1876702015021312:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762830.597645:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2364:x1876702015021312:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2390us (251621us total) trans 4294967924 rc 0/0 00000100:00100000:1.0:1789762830.597655:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 715 00002000:00100000:1.0:1789762830.597664:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x13:0x0] (5242880->6291455) req@ffff981ebfdbad80 x1876702015021312/t4294967924(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.597675:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbad80 with x1876702015021312 ext(5242880->6291455) 00002000:00010000:1.0:1789762830.597681:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdbad80 00000100:00100000:2.0:1789762830.645358:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353999252802 time 353949252802 state 0x1 ref 7 00000100:00100000:2.0:1789762830.645383:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2365:x1876702015021440:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762830.648499:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:2365:x1876702015021440:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3123us (291409us total) trans 4294967405 rc 0/0 00000100:00100000:1.0:1789762830.648509:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 716 00002000:00100000:1.0:1789762830.648519:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xd:0x0] (7340032->8388607) req@ffff981ebfdbb800 x1876702015021440/t4294967405(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.648527:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbb800 with x1876702015021440 ext(7340032->8388607) 00002000:00010000:1.0:1789762830.648533:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdbb800 00000100:00100000:3.0:1789762830.650873:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.650930:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015022592 00000100:00100000:1.0:1789762830.650938:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015022592 00000100:00100000:1.0:1789762830.650979:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 86 00000100:00100000:1.0:1789762830.650985:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:14237:x1876702015022592:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:1.0:1789762830.651227:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:14237:x1876702015022592:12345-192.168.203.48@tcp:35:dd.0 Request processed in 241us (355us total) trans 4294967532 rc 0/0 00000100:00100000:1.0:1789762830.651233:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 86 00000100:00100000:3.0:1789762830.651525:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762830.651560:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015022336 00000100:00100000:0.0:1789762830.651566:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015022336 00002000:00100000:0.0:1789762830.651592:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0xd:0x0] (6291456->7340031) req@ffff981ebfdbbb80 x1876702015022336/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762830.651600:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbbb80 with x1876702015022336 ext(6291456->7340031) 00002000:00010000:0.0:1789762830.651603:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdbbb80 00000100:00100000:0.0:1789762830.651616:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353999252802 time 353949252802 state 0x1 ref 7 00000100:00100000:3.0:1789762830.651722:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762830.651758:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015022720 00000100:00100000:1.0:1789762830.651763:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015022720 00002000:00100000:1.0:1789762830.651785:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xd:0x0] (3145728->4194303) req@ffff981ebfdba300 x1876702015022720/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/0 e 0 to 0 dl 1789762841 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.651796:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdba300 with x1876702015022720 ext(3145728->4194303) 00002000:00010000:1.0:1789762830.651800:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdba300 00000100:00100000:1.0:1789762830.651813:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 353999252802 time 353949252802 state 0x1 ref 8 00000100:00100000:0.0:1789762830.690595:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 354094484811 time 353994484811 state 0x1 ref 5 00000100:00100000:0.0:1789762830.690627:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2364:x1876702015019136:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762830.693427:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2364:x1876702015019136:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2813us (637877us total) trans 4294967925 rc 0/0 00000100:00100000:0.0:1789762830.693435:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 704 00002000:00100000:0.0:1789762830.693445:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x240000400:0x12:0x0] (7340032->8388607) req@ffff981ebac58a80 x1876702015019136/t4294967925(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762830.693453:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac58a80 with x1876702015019136 ext(7340032->8388607) 00002000:00010000:0.0:1789762830.693459:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac58a80 00000100:00100000:1.0:1789762830.695411:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 354049314833 time 353999314833 state 0x1 ref 8 00000100:00100000:1.0:1789762830.695428:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2365:x1876702015021568:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762830.697653:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2365:x1876702015021568:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2227us (339796us total) trans 4294967406 rc 0/0 00000100:00100000:2.0:1789762830.697661:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 717 00002000:00100000:2.0:1789762830.697668:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xd:0x0] (1048576->2097151) req@ffff981ebac5a680 x1876702015021568/t4294967406(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762830.697675:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac5a680 with x1876702015021568 ext(1048576->2097151) 00002000:00010000:2.0:1789762830.697679:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac5a680 00000100:00100000:1.0:1789762830.745500:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 354099395369 time 354049395369 state 0x1 ref 7 00000100:00100000:1.0:1789762830.745523:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2363:x1876702015021696:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762830.748341:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2363:x1876702015021696:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2826us (353619us total) trans 4294967926 rc 0/0 00000100:00100000:1.0:1789762830.748349:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 718 00002000:00100000:1.0:1789762830.748358:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x13:0x0] (6291456->7340031) req@ffff981ebac59180 x1876702015021696/t4294967926(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.748367:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac59180 with x1876702015021696 ext(6291456->7340031) 00002000:00010000:1.0:1789762830.748373:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac59180 00000100:00100000:1.0:1789762830.790658:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 354194553709 time 354094553709 state 0x1 ref 4 00000100:00100000:1.0:1789762830.790682:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2364:x1876702015019264:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762830.796911:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:2364:x1876702015019264:12345-192.168.203.48@tcp:4:dd.500 Request processed in 6236us (701887us total) trans 4294967407 rc 0/0 00000100:00100000:1.0:1789762830.796923:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 705 00002000:00100000:1.0:1789762830.796936:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xc:0x0] (4194304->5242879) req@ffff981ebfdb8e00 x1876702015019264/t4294967407(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762830.796948:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb8e00 with x1876702015019264 ext(4194304->5242879) 00002000:00010000:1.0:1789762830.796954:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdb8e00 00000100:00100000:1.0:1789762830.796981:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 354150887898 time 354100887898 state 0x1 ref 6 00000100:00100000:1.0:1789762830.796991:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2364:x1876702015021824:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762830.799104:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2364:x1876702015021824:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2094us (400298us total) trans 4294967408 rc 0/0 00000100:00100000:1.0:1789762830.799110:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 719 00002000:00100000:1.0:1789762830.799117:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xd:0x0] (4194304->5242879) req@ffff981ebfdb8380 x1876702015021824/t4294967408(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.799123:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb8380 with x1876702015021824 ext(4194304->5242879) 00002000:00010000:1.0:1789762830.799127:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdb8380 00000100:00100000:1.0:1789762830.847052:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 354200947553 time 354150947553 state 0x1 ref 5 00000100:00100000:1.0:1789762830.847076:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:2363:x1876702015021952:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762830.849796:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:2363:x1876702015021952:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2727us (450795us total) trans 4294967409 rc 0/0 00000100:00100000:1.0:1789762830.849803:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 720 00002000:00100000:1.0:1789762830.849813:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xd:0x0] (5242880->6291455) req@ffff981ebfdbb480 x1876702015021952/t4294967409(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.849821:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbb480 with x1876702015021952 ext(5242880->6291455) 00002000:00010000:1.0:1789762830.849826:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdbb480 00000100:00100000:0.0:1789762830.890791:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 354294680591 time 354194680591 state 0x1 ref 3 00000100:00100000:0.0:1789762830.890822:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2364:x1876702015019392:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762830.893920:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2364:x1876702015019392:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3109us (791998us total) trans 4294967410 rc 0/0 00000100:00100000:0.0:1789762830.893931:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 706 00002000:00100000:0.0:1789762830.893947:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0xc:0x0] (5242880->6291455) req@ffff981ebac58000 x1876702015019392/t4294967410(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762830.893958:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac58000 with x1876702015019392 ext(5242880->6291455) 00002000:00010000:0.0:1789762830.893965:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac58000 00000100:00100000:2.0:1789762830.897113:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 354251017008 time 354201017008 state 0x1 ref 4 00000100:00100000:2.0:1789762830.897138:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2366:x1876702015022080:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762830.899704:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2366:x1876702015022080:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2578us (495888us total) trans 4294967927 rc 0/0 00000100:00100000:0.0:1789762830.899714:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 721 00002000:00100000:0.0:1789762830.899722:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x13:0x0] (7340032->8388607) req@ffff981ebfdb9c00 x1876702015022080/t4294967927(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762830.899747:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb9c00 with x1876702015022080 ext(7340032->8388607) 00002000:00010000:0.0:1789762830.899752:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdb9c00 00000100:00100000:1.0:1789762830.947229:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 354301122958 time 354251122958 state 0x1 ref 3 00000100:00100000:1.0:1789762830.947252:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2366:x1876702015022208:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762830.950181:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2366:x1876702015022208:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2936us (500601us total) trans 4294967928 rc 0/0 00000100:00100000:1.0:1789762830.950189:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 722 00002000:00100000:1.0:1789762830.950199:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x13:0x0] (4194304->5242879) req@ffff981ebfdb9f80 x1876702015022208/t4294967928(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762830.950207:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb9f80 with x1876702015022208 ext(4194304->5242879) 00002000:00010000:1.0:1789762830.950212:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdb9f80 00000100:00100000:0.0:1789762830.990875:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 354394771489 time 354294771489 state 0x1 ref 2 00000100:00100000:0.0:1789762830.990901:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:2364:x1876702015019520:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762830.993998:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2364:x1876702015019520:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3106us (798275us total) trans 4294967411 rc 0/0 00000100:00100000:0.0:1789762830.994005:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 707 00002000:00100000:0.0:1789762830.994015:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xc:0x0] (6291456->7340031) req@ffff981ebfdba680 x1876702015019520/t4294967411(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762830.994022:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdba680 with x1876702015019520 ext(6291456->7340031) 00002000:00010000:0.0:1789762830.994028:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdba680 00000100:00100000:0.0:1789762830.997270:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 354351174380 time 354301174380 state 0x1 ref 2 00000100:00100000:0.0:1789762830.997282:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2363:x1876702015022336:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762831.002817:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2363:x1876702015022336:12345-192.168.203.48@tcp:4:dd.0 Request processed in 5535us (351290us total) trans 4294967412 rc 0/0 00000100:00100000:0.0:1789762831.002823:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 723 00002000:00100000:0.0:1789762831.002829:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0xd:0x0] (6291456->7340031) req@ffff981ebfdbbb80 x1876702015022336/t4294967412(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762831.002836:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbbb80 with x1876702015022336 ext(6291456->7340031) 00002000:00010000:0.0:1789762831.002845:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdbbb80 00000100:00100000:2.0:1789762831.047356:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 354351174380 time 354351252788 state 0x0 ref 1 00000100:00100000:2.0:1789762831.047395:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2363:x1876702015022720:12345-192.168.203.48@tcp:4:dd.0 00000100:00080000:1.0:1789762831.047453:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762831.050457:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2363:x1876702015022720:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3084us (398734us total) trans 4294967413 rc 0/0 00000100:00100000:1.0:1789762831.050466:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 724 00000100:00100000:1.0:1789762831.050474:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 354351174380 time 354351252788 state 0x2 ref 0 00002000:00100000:1.0:1789762831.050486:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xd:0x0] (3145728->4194303) req@ffff981ebfdba300 x1876702015022720/t4294967413(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762831.050493:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdba300 with x1876702015022720 ext(3145728->4194303) 00002000:00010000:1.0:1789762831.050499:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdba300 00000100:00100000:1.0:1789762831.090953:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 354394771489 time 354394843948 state 0x0 ref 1 00000100:00100000:1.0:1789762831.090979:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2364:x1876702015019648:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762831.094216:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2364:x1876702015019648:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3244us (890890us total) trans 4294967414 rc 0/0 00000100:00100000:1.0:1789762831.094227:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 708 00000100:00100000:1.0:1789762831.094239:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 354394771489 time 354394843948 state 0x2 ref 0 00002000:00100000:1.0:1789762831.094254:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0xc:0x0] (7340032->8388607) req@ffff981ebac59500 x1876702015019648/t4294967414(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:611/0 lens 488/448 e 0 to 0 dl 1789762841 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762831.094265:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac59500 with x1876702015019648 ext(7340032->8388607) 00002000:00010000:1.0:1789762831.094274:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebac59500 00000100:00100000:2.0:1789762831.132877:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.132949:0:8405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015022848 00000100:00100000:1.0:1789762831.132959:0:8405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015022848 00000100:00100000:1.0:1789762831.133021:0:8405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 23 00000100:00100000:1.0:1789762831.133032:0:8405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:5593:x1876702015022848:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762831.133054:0:8405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00002000:00010000:1.0:1789762831.133075:0:8405:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x12:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:1.0:1789762831.133079:0:8405:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x12:0x0] updating lvb mtime from disk: 0 -> 1789762830 00002000:00010000:1.0:1789762831.133082:0:8405:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x12:0x0] updating lvb ctime from disk: 0 -> 1789762830 00010000:00010000:1.0:1789762831.133150:0:8405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003821af31/0x416193e012b04353 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd62b expref: 39 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762831.133188:0:8405:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:1.0:1789762831.133191:0:8405:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb mtime from disk: 0 -> 1789762830 00002000:00010000:1.0:1789762831.133194:0:8405:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb ctime from disk: 0 -> 1789762830 00010000:00010000:1.0:1789762831.133205:0:8405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000481d0182/0x416193e012b04392 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd647 expref: 38 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.133222:0:8405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762831.133241:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762831.133279:0:5792:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015022976 00000100:00100000:1.0:1789762831.133285:0:8405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:5593:x1876702015022848:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 250us (413us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762831.133286:0:5792:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015022976 00000100:00100000:1.0:1789762831.133301:0:8405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 23 00000100:00100000:0.0:1789762831.133334:0:5792:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 24 00000100:00100000:0.0:1789762831.133345:0:5792:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:6050:x1876702015022976:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762831.133354:0:5792:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00002000:00010000:0.0:1789762831.133366:0:5792:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xc:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:0.0:1789762831.133368:0:5792:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xc:0x0] updating lvb mtime from disk: 0 -> 1789762830 00002000:00010000:0.0:1789762831.133370:0:5792:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xc:0x0] updating lvb ctime from disk: 0 -> 1789762830 00002000:00010000:0.0:1789762831.133372:0:5792:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xc:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:0.0:1789762831.133384:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004e67eb4a/0x416193e012b04361 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd632 expref: 37 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762831.133404:0:5792:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xd:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:0.0:1789762831.133406:0:5792:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xd:0x0] updating lvb mtime from disk: 0 -> 1789762830 00002000:00010000:0.0:1789762831.133408:0:5792:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xd:0x0] updating lvb ctime from disk: 0 -> 1789762830 00010000:00010000:0.0:1789762831.133413:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003cb287b8/0x416193e012b0438b lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd640 expref: 36 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.133422:0:5792:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762831.133454:0:5792:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:6050:x1876702015022976:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 111us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762831.133461:0:5792:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 24 00000100:00100000:3.0:1789762831.163219:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.163277:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015023232 00000100:00100000:1.0:1789762831.163285:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015023232 00000100:00100000:3.0:1789762831.163290:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.163328:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015023488 00000100:00100000:1.0:1789762831.163332:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015023488 00000100:00100000:3.0:1789762831.163340:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 280 00000100:00100000:3.0:1789762831.163352:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:14263:x1876702015023232:12345-192.168.203.48@tcp:101:dd.0 00000100:00100000:2.0:1789762831.163359:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 281 00000100:00100000:2.0:1789762831.163378:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:14264:x1876702015023488:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762831.163392:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762831.163393:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762831.163411:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04399 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd64e expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.163417:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b043a0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd655 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.163510:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b043a7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.163513:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ba00/0x416193e012b043ae lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.163534:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ba00/0x416193e012b043ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.163550:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ba00/0x416193e012b043ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.163551:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b043a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.163572:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b043a7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762831.163576:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2e:0x0]: returning size 16777216 00000004:00200000:2.0:1789762831.163582:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2e:0x0]: returning size 16777216 00010000:00010000:3.0:1789762831.163725:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063b200/0x416193e012b04399 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd64e expref: 26 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.163729:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b043a0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd655 expref: 26 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.163748:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ba00/0x416193e012b043ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd64e expref: 24 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.163752:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b043a7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd655 expref: 24 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.163766:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2e:0x0].0x0 (ffff981e9272c000) 00010000:00010000:3.0:1789762831.163772:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762831.163776:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e9063ba00, rc 0) 00010000:00010000:2.0:1789762831.163778:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2e:0x0].0x0 (ffff981e9272c000) 00010000:00010000:2.0:1789762831.163782:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789762831.163789:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87c1a400, rc 0) 00000100:00100000:3.0:1789762831.163829:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:14263:x1876702015023232:12345-192.168.203.48@tcp:101:dd.0 Request processed in 476us (611us total) trans 4294967533 rc 0/0 00000100:00100000:2.0:1789762831.163831:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:14264:x1876702015023488:12345-192.168.203.48@tcp:101:dd.0 Request processed in 455us (569us total) trans 4294967534 rc 0/0 00000100:00100000:3.0:1789762831.163844:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 280 00000100:00100000:2.0:1789762831.163853:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 281 00000100:00100000:2.0:1789762831.164279:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762831.164305:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015023744 00000100:00100000:3.0:1789762831.164310:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015023744 00000100:00100000:3.0:1789762831.164341:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 282 00000100:00100000:3.0:1789762831.164355:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:14265:x1876702015023744:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762831.164364:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762831.164372:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b043c3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd65c expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.164406:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063a400/0x416193e012b043ca lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.164413:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063a400/0x416193e012b043ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.164419:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063a400/0x416193e012b043ca lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762831.164425:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2e:0x0]: returning size 16777216 00010000:00010000:3.0:1789762831.164485:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ac00/0x416193e012b043c3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd65c expref: 26 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.164496:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063a400/0x416193e012b043ca lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd65c expref: 25 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.164501:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2e:0x0].0x0 (ffff981e9272c000) 00010000:00010000:3.0:1789762831.164503:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762831.164505:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e9063a400, rc 0) 00000100:00100000:3.0:1789762831.164538:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:14265:x1876702015023744:12345-192.168.203.48@tcp:101:dd.0 Request processed in 183us (258us total) trans 4294967535 rc 0/0 00000100:00100000:3.0:1789762831.164546:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 282 00000100:00100000:0.0:1789762831.165111:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.165179:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015023872 00000100:00100000:1.0:1789762831.165185:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015023872 00000100:00100000:1.0:1789762831.165218:0:6664:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 25 00000100:00100000:1.0:1789762831.165225:0:6664:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2364:x1876702015023872:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762831.165233:0:6664:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762831.165245:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063ba00/0x416193e012b043ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd64e expref: 25 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.165256:0:6664:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2e:0x0].0x0 (ffff981e9272c000) 00010000:00010000:1.0:1789762831.165258:0:6664:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762831.165284:0:6664:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dd880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2364:x1876702015023872:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 60us (199us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762831.165289:0:6664:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 25 00000100:00100000:2.0:1789762831.165372:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.165403:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015024000 00000100:00100000:1.0:1789762831.165408:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015024000 00000100:00100000:1.0:1789762831.165433:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 177 00000100:00100000:1.0:1789762831.165439:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:14263:x1876702015024000:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762831.165445:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762831.165458:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b043d8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd663 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762831.165480:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x12:0x0] initial LVB size: 8388608, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x1 00010000:00010000:1.0:1789762831.165495:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=354s v=(0 0) 00010000:00010000:1.0:1789762831.165501:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.165511:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b043d8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd663 expref: 38 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.165526:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b043d8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd663 expref: 38 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.165535:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81483c00, rc 0) 00000100:00100000:0.0:1789762831.165541:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.165568:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:14263:x1876702015024000:12345-192.168.203.48@tcp:101:dd.0 Request processed in 128us (196us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762831.165575:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 177 00000100:00100000:2.0:1789762831.165580:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015024128 00000100:00100000:2.0:1789762831.165586:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015024128 00000100:00100000:2.0:1789762831.165625:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 26 00000100:00100000:2.0:1789762831.165635:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2363:x1876702015024128:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762831.165645:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762831.165654:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b043a7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd655 expref: 24 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.165669:0:5790:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2e:0x0].0x0 (ffff981e9272c000) 00010000:00010000:2.0:1789762831.165672:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762831.165700:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2363:x1876702015024128:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 68us (162us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762831.165705:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 26 00000100:00100000:3.0:1789762831.165756:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.165791:0:7595:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015024256 00000100:00100000:1.0:1789762831.165795:0:7595:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015024256 00000100:00100000:1.0:1789762831.165821:0:7595:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 27 00000100:00100000:1.0:1789762831.165827:0:7595:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86f03450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2364:x1876702015024256:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762831.165834:0:7595:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762831.165843:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063a400/0x416193e012b043ca lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd65c expref: 23 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.165852:0:7595:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2e:0x0].0x0 (ffff981e9272c000) 00010000:00010000:1.0:1789762831.165855:0:7595:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762831.165876:0:7595:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86f03450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:2364:x1876702015024256:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 50us (133us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762831.165882:0:7595:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 27 00000100:00100000:0.0:1789762831.165932:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.165962:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015024512 00000100:00100000:1.0:1789762831.165966:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015024512 00000100:00100000:1.0:1789762831.165996:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 87 00000100:00100000:1.0:1789762831.166002:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:14264:x1876702015024512:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:1.0:1789762831.166119:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:14264:x1876702015024512:12345-192.168.203.48@tcp:35:dd.0 Request processed in 117us (189us total) trans 4294967536 rc 0/0 00000100:00100000:1.0:1789762831.166141:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 87 00000100:00100000:3.0:1789762831.169484:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.169529:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015024640 00000100:00100000:1.0:1789762831.169535:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015024640 00002000:00100000:1.0:1789762831.169558:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x12:0x0] (0->1048575) req@ffff981ebc49c000 x1876702015024640/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:612/0 lens 488/0 e 0 to 0 dl 1789762842 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762831.169566:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49c000 with x1876702015024640 ext(0->1048575) 00002000:00010000:1.0:1789762831.169570:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc49c000 00000100:00100000:1.0:1789762831.169583:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323368356307 time 323368290950 state 0x0 ref 1 00000100:00100000:1.0:1789762831.169590:0:6654:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 8: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 323368356307 time 323368290950 state 0x0 ref 1 00000100:00100000:1.0:1789762831.169602:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 3 deadline 354573509327 time 354473509327 state 0x1 ref 1 00000100:00100000:1.0:1789762831.169623:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 354573509327 time 354473532209 state 0x0 ref 1 00000100:00100000:1.0:1789762831.169632:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702015024640:12345-192.168.203.48@tcp:3:dd.0 00000100:00100000:0.0:1789762831.169940:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762831.169992:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015024896 00000100:00100000:2.0:1789762831.169998:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015024896 00000100:00100000:2.0:1789762831.170040:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 88 00000100:00100000:2.0:1789762831.170051:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b18850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:14265:x1876702015024896:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:2.0:1789762831.170252:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b18850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:14265:x1876702015024896:12345-192.168.203.48@tcp:35:dd.0 Request processed in 202us (310us total) trans 4294967537 rc 0/0 00000100:00100000:2.0:1789762831.170262:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 88 00000100:00100000:1.0:1789762831.173408:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702015024640:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3775us (3925us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762831.173416:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 725 00000100:00100000:1.0:1789762831.173425:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 354573509327 time 354473532209 state 0x2 ref 0 00002000:00100000:1.0:1789762831.173435:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x12:0x0] (0->1048575) req@ffff981ebc49c000 x1876702015024640/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:612/0 lens 488/440 e 0 to 0 dl 1789762842 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762831.173443:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49c000 with x1876702015024640 ext(0->1048575) 00002000:00010000:1.0:1789762831.173447:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc49c000 00000100:00100000:0.0:1789762831.202341:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.202418:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015025152 00000100:00100000:1.0:1789762831.202426:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015025152 00000100:00100000:1.0:1789762831.202469:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 89 00000100:00100000:1.0:1789762831.202478:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:14275:x1876702015025152:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:1.0:1789762831.202636:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:14275:x1876702015025152:12345-192.168.203.48@tcp:35:dd.0 Request processed in 159us (299us total) trans 4294967538 rc 0/0 00000100:00100000:1.0:1789762831.202642:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 89 00000100:00100000:3.0:1789762831.213004:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762831.213055:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015025280 00000100:00100000:0.0:1789762831.213062:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015025280 00000100:00100000:0.0:1789762831.213123:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 283 00000100:00100000:0.0:1789762831.213147:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:14279:x1876702015025280:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:0.0:1789762831.213160:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762831.213176:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfa00/0x416193e012b043df lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd66a expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.213219:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf600/0x416193e012b043e6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.213230:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf600/0x416193e012b043e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.213239:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf600/0x416193e012b043e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762831.213259:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 16384 00000004:00200000:0.0:1789762831.213261:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762831.213272:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf600/0x416193e012b043e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.213289:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfa00/0x416193e012b043df lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd66a expref: 27 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.213301:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf600/0x416193e012b043e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd66a expref: 26 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.213307:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:0.0:1789762831.213321:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762831.213324:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbf600, rc 0) 00000100:00100000:0.0:1789762831.213354:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:14279:x1876702015025280:12345-192.168.203.48@tcp:101:rm.0 Request processed in 210us (354us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762831.213359:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 283 00000100:00100000:3.0:1789762831.214390:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762831.214442:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015025536 00000100:00100000:2.0:1789762831.214449:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015025536 00000100:00100000:2.0:1789762831.214487:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 284 00000100:00100000:2.0:1789762831.214497:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:14279:x1876702015025536:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:2.0:1789762831.214507:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762831.214525:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b043ed lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd671 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762831.214575:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 16384 00010000:00010000:2.0:1789762831.214647:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b043ed lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd671 expref: 28 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.214660:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:2.0:1789762831.214670:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762831.214675:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1be00/0x416193e012b043ed lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd671 expref: 28 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.214684:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87c1be00, rc 0) 00000100:00100000:2.0:1789762831.214713:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:14279:x1876702015025536:12345-192.168.203.48@tcp:101:rm.0 Request processed in 219us (325us total) trans 4294967539 rc 301/301 00000100:00100000:2.0:1789762831.214718:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 284 00000100:00100000:2.0:1789762831.216121:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.216170:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015025664 00000100:00100000:1.0:1789762831.216175:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015025664 00000100:00100000:1.0:1789762831.216205:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 90 00000100:00100000:1.0:1789762831.216212:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:14279:x1876702015025664:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:1.0:1789762831.217852:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:14279:x1876702015025664:12345-192.168.203.48@tcp:37:rm.0 Request processed in 1638us (1729us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762831.217858:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 90 00000100:00100000:3.0:1789762831.219277:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.219326:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015025920 00000100:00100000:1.0:1789762831.219332:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015025920 00000100:00100000:1.0:1789762831.219358:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 91 00000100:00100000:1.0:1789762831.219365:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:14279:x1876702015025920:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:1.0:1789762831.219452:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:14279:x1876702015025920:12345-192.168.203.48@tcp:35:rm.0 Request processed in 86us (175us total) trans 4294967540 rc 0/0 00000100:00100000:1.0:1789762831.219462:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 91 00000100:00100000:2.0:1789762831.220272:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762831.220317:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015026176 00000100:00100000:3.0:1789762831.220323:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015026176 00000100:00100000:3.0:1789762831.220359:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 285 00000100:00100000:3.0:1789762831.220370:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:14279:x1876702015026176:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:3.0:1789762831.220384:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762831.220395:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b043fb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd678 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.220448:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04402 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.220461:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04402 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.220473:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04402 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762831.220482:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 16384 00010000:00010000:3.0:1789762831.220553:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b043fb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd678 expref: 31 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.220570:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04402 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd678 expref: 30 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.220579:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:3.0:1789762831.220582:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1789762831.220593:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb4c00, rc 0) 00000100:00100000:3.0:1789762831.220632:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14279:x1876702015026176:12345-192.168.203.48@tcp:101:rm.0 Request processed in 263us (360us total) trans 4294967541 rc 0/0 00000100:00100000:3.0:1789762831.220641:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 285 00000100:00100000:3.0:1789762831.221659:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762831.221706:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015026304 00000100:00100000:0.0:1789762831.221712:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015026304 00000100:00100000:0.0:1789762831.221742:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 286 00000100:00100000:0.0:1789762831.221748:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14279:x1876702015026304:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:0.0:1789762831.221780:0:5804:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7719616512 avail 7715422208 00000100:00100000:0.0:1789762831.221813:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14279:x1876702015026304:12345-192.168.203.48@tcp:41:rm.0 Request processed in 65us (155us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762831.221819:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 286 00000100:00100000:0.0:1789762831.222700:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762831.222740:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015026560 00000100:00100000:2.0:1789762831.222746:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015026560 00000100:00100000:2.0:1789762831.222782:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 287 00000100:00100000:2.0:1789762831.222794:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14279:x1876702015026560:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:2.0:1789762831.222820:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1789762831.222834:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063be00/0x416193e012b04345 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd624 expref: 30 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.222844:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2e:0x0].0x0 (ffff981e9272c000) 00010000:00010000:2.0:1789762831.222854:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf600/0x416193e012b043e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd66a expref: 29 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.222861:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:2.0:1789762831.222865:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762831.222877:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a600/0x416193e012b04410 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.222887:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a600/0x416193e012b04410 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.222904:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a600/0x416193e012b04410 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.222920:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b000/0x416193e012b04417 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.222926:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b000/0x416193e012b04417 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.222930:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b000/0x416193e012b04417 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.222989:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b200/0x416193e012b0441e lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.222995:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b200/0x416193e012b0441e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223000:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b200/0x416193e012b0441e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789762831.223450:0:8165:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].18, 64 off9344 00000004:00080000:2.0:1789762831.223457:0:8165:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 354527369656ns, next at 355527368324ns, rc = 0 00000040:00080000:2.0:1789762831.223509:0:8165:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].12, 64 off8960 00000004:00080000:2.0:1789762831.223517:0:8165:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 354527429539ns, next at 355527428233ns, rc = 0 00010000:00010000:2.0:1789762831.223563:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b04425 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223571:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b04425 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223585:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b04425 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223590:0:8165:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b04425 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223594:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b04425 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223598:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b04425 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223613:0:8165:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762831.223624:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b200/0x416193e012b0441e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223628:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b200/0x416193e012b0441e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.223632:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762831.223633:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b200/0x416193e012b0441e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.223640:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b04425 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223642:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762831.223648:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b200/0x416193e012b0441e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.223651:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762831.223656:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b200/0x416193e012b0441e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.223657:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b04425 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.223678:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b04425 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223678:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2e:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762831.223685:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762831.223698:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b200/0x416193e012b0441e lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.223704:0:5793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2e:0x0].0x0 (ffff981e82a26b00) 00010000:00010000:2.0:1789762831.223704:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b200/0x416193e012b0441e lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.223713:0:5793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789762831.223719:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b04425 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223722:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a600/0x416193e012b04410 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.223734:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a400/0x416193e012b04425 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223735:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a600/0x416193e012b04410 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223754:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a600/0x416193e012b04410 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223761:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762831.223763:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762831.223766:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a600/0x416193e012b04410 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.223767:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762831.223770:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762831.223779:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a600/0x416193e012b04410 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223790:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:2.0:1789762831.223793:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762831.223797:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a600/0x416193e012b04410 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.223803:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a600/0x416193e012b04410 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762831.223814:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009647120a reply state 000000008e4e73dc transno 4294967542 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b000/0x416193e012b04417 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762831.223834:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762831.223883:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902ddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14279:x1876702015026560:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1092us (1182us total) trans 4294967542 rc 0/0 00000100:00100000:2.0:1789762831.223891:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 287 00010000:00010000:1.0:1789762831.224444:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b000/0x416193e012b04417 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.224451:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b000/0x416193e012b04417 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.224456:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b000/0x416193e012b04417 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.224463:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762831.224465:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b000/0x416193e012b04417 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.224470:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b000/0x416193e012b04417 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.224477:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x77289514 (ffff981e82a26500) 00010000:00010000:1.0:1789762831.224480:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762831.224483:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b000/0x416193e012b04417 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.224489:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b000/0x416193e012b04417 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.226106:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762831.226179:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015026816 00000100:00100000:3.0:1789762831.226184:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015026816 00000100:00100000:3.0:1789762831.226219:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 288 00000100:00100000:3.0:1789762831.226228:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:14279:x1876702015026816:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:3.0:1789762831.226254:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762831.226267:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a200/0x416193e012b0437d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd639 expref: 29 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226276:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff981e82a26600) 00010000:00010000:3.0:1789762831.226281:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762831.226288:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b0442c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226294:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b0442c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226299:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b0442c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226309:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04433 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226313:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04433 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226327:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04433 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226365:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b0443a lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226371:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b0443a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226376:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b0443a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762831.226615:0:5806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].13, 64 off9024 00000040:00080000:3.0:1789762831.226627:0:5806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].19, 64 off9408 00010000:00010000:3.0:1789762831.226659:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04441 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226666:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04441 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226672:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04441 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226676:0:5806:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04441 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226681:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04441 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226684:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04441 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226712:0:5806:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762831.226723:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b0443a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226730:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b0443a lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.226735:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762831.226738:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b0443a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.226740:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04441 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226745:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762831.226747:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762831.226748:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b0443a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.226750:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04441 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226756:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b0443a lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.226758:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04441 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226768:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff981e9272c800) 00010000:00010000:3.0:1789762831.226770:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762831.226774:0:8466:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff981e9272c800) 00010000:00010000:3.0:1789762831.226774:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b0443a lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.226777:0:8466:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789762831.226780:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04441 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226781:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b0443a lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.226792:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04441 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226794:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b0442c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226801:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b0442c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.226805:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762831.226806:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762831.226808:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762831.226818:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b0442c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226824:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762831.226827:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b0442c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226834:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b0442c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226840:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:3.0:1789762831.226842:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762831.226844:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b0442c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.226848:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b0442c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762831.226855:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000053b33e2d reply state 0000000054e3959d transno 4294967543 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04433 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762831.226859:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762831.226893:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902df800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14279:x1876702015026816:12345-192.168.203.48@tcp:36:rm.0 Request processed in 668us (810us total) trans 4294967543 rc 0/0 00000100:00100000:3.0:1789762831.226899:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 288 00010000:00010000:3.0:1789762831.227423:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04433 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.227430:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04433 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.227444:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04433 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.227448:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762831.227450:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04433 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.227454:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04433 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.227461:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x992ad74f (ffff981e9272c300) 00010000:00010000:3.0:1789762831.227463:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762831.227465:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04433 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.227470:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04433 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.227955:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.227988:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015027072 00000100:00100000:1.0:1789762831.227992:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015027072 00000100:00100000:1.0:1789762831.228019:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 92 00000100:00100000:1.0:1789762831.228028:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:14279:x1876702015027072:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:1.0:1789762831.228195:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14279:x1876702015027072:12345-192.168.203.48@tcp:35:rm.0 Request processed in 168us (240us total) trans 4294967544 rc 0/0 00000100:00100000:1.0:1789762831.228201:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 92 00000100:00100000:2.0:1789762831.228916:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762831.228954:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015027328 00000100:00100000:0.0:1789762831.228959:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015027328 00000100:00100000:0.0:1789762831.228986:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 289 00000100:00100000:0.0:1789762831.228993:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14279:x1876702015027328:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:0.0:1789762831.229013:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1789762831.229023:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04402 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd678 expref: 30 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229037:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482800/0x416193e012b04329 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd61d expref: 29 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229061:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:0.0:1789762831.229076:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e9063a600/0x416193e012b042c7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd608 expref: 28 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229114:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:0.0:1789762831.229121:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762831.229152:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe800/0x416193e012b04448 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229163:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe800/0x416193e012b04448 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229173:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe800/0x416193e012b04448 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229186:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b0444f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229193:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b0444f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229199:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b0444f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229248:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04456 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229254:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04456 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229267:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04456 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762831.229523:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 16384 00010000:00010000:0.0:1789762831.229534:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04456 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229541:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04456 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229548:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04456 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229554:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbfc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762831.229558:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04456 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229565:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04456 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229575:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff981ea0aa3500) 00010000:00010000:0.0:1789762831.229578:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762831.229582:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04456 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229587:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbfc00/0x416193e012b04456 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229598:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe800/0x416193e012b04448 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229602:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe800/0x416193e012b04448 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229616:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe800/0x416193e012b04448 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229622:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762831.229626:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe800/0x416193e012b04448 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229633:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe800/0x416193e012b04448 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229643:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3c00) 00010000:00010000:0.0:1789762831.229646:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762831.229649:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe800/0x416193e012b04448 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.229656:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe800/0x416193e012b04448 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762831.229667:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001a7af6e4 reply state 000000002eb8ed3f transno 4294967545 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b0444f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.229673:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762831.229714:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:14279:x1876702015027328:12345-192.168.203.48@tcp:36:rm.0 Request processed in 721us (799us total) trans 4294967545 rc 0/0 00000100:00100000:0.0:1789762831.229720:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 289 00010000:00010000:1.0:1789762831.230264:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b0444f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.230273:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b0444f lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.230278:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b0444f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.230282:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762831.230284:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b0444f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.230289:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b0444f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.230299:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3a00) 00010000:00010000:1.0:1789762831.230301:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762831.230303:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b0444f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.230308:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf400/0x416193e012b0444f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.288186:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762831.519542:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762831.519594:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015027456 00000100:00100000:3.0:1789762831.519606:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015027456 00000100:00100000:3.0:1789762831.519661:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 290 00000100:00100000:3.0:1789762831.519671:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:14290:x1876702015027456:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:3.0:1789762831.519691:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762831.519709:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b0445d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd67f expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.519760:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04464 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.519781:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04464 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.519789:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04464 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762831.519818:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762831.519821:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762831.519834:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04464 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.519852:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b0445d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd67f expref: 30 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.519863:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04464 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd67f expref: 29 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.519869:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762831.519872:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762831.519875:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb4800, rc 0) 00000100:00100000:3.0:1789762831.519922:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:14290:x1876702015027456:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 254us (386us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762831.519928:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 290 00000100:00100000:2.0:1789762831.520942:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762831.520973:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015027712 00000100:00100000:3.0:1789762831.520978:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015027712 00000100:00100000:3.0:1789762831.521007:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 291 00000100:00100000:3.0:1789762831.521014:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:14290:x1876702015027712:12345-192.168.203.48@tcp:36:mkdir.0 00010000:00010000:3.0:1789762831.521055:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762831.521078:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b04464 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd67f expref: 29 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521116:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762831.521137:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762831.521189:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b0446b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521197:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b0446b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521203:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b0446b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521212:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04472 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521217:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04472 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521223:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04472 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762831.521763:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00010000:00010000:3.0:1789762831.521771:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b0446b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521777:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b0446b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521782:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b0446b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521793:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762831.521797:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b0446b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521803:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b0446b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521811:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:3.0:1789762831.521814:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762831.521817:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b0446b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.521821:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b0446b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762831.521831:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000006534d3d reply state 00000000f1698db8 transno 4294967546 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04472 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762831.521838:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762831.521884:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14290:x1876702015027712:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 871us (943us total) trans 4294967546 rc 0/0 00000100:00100000:3.0:1789762831.521889:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 291 00010000:00010000:1.0:1789762831.522409:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04472 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.522418:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04472 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.522423:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04472 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.522427:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762831.522430:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04472 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.522434:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04472 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.522442:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272cc00) 00010000:00010000:1.0:1789762831.522444:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762831.522447:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04472 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.522452:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04472 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762831.527415:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.527450:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015027840 00000100:00100000:1.0:1789762831.527455:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015027840 00000100:00100000:1.0:1789762831.527490:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 292 00000100:00100000:1.0:1789762831.527499:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:14291:x1876702015027840:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762831.527512:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762831.527526:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b04479 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd686 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.527562:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b04480 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.527570:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b04480 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.527577:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b04480 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762831.527593:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762831.527595:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762831.527606:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b04480 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.527620:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483600/0x416193e012b04479 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd686 expref: 31 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.527631:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b04480 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd686 expref: 30 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.527637:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:1.0:1789762831.527639:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762831.527641:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81482c00, rc 0) 00000100:00100000:1.0:1789762831.527674:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14291:x1876702015027840:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 178us (262us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762831.527680:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 292 00000100:00100000:3.0:1789762831.528620:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762831.528655:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015028096 00000100:00100000:2.0:1789762831.528661:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015028096 00000100:00100000:2.0:1789762831.528705:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 293 00000100:00100000:2.0:1789762831.528718:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14291:x1876702015028096:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:2.0:1789762831.528729:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762831.528731:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1789762831.528747:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b04480 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd686 expref: 30 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528760:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:2.0:1789762831.528766:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762831.528776:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a000/0x416193e012b04487 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd68d expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528854:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0448e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528862:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0448e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528874:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0448e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528884:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b400/0x416193e012b04495 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528889:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b400/0x416193e012b04495 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528894:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b400/0x416193e012b04495 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528944:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b0449c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528950:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b0449c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528955:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b0449c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762831.528961:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00010000:00010000:2.0:1789762831.528968:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b0449c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528973:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b0449c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528978:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b0449c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528981:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762831.528989:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b0449c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.528995:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b0449c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529002:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981e82a26000) 00010000:00010000:2.0:1789762831.529004:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762831.529006:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b0449c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529010:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b0449c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529023:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0448e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529027:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0448e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529032:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0448e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529036:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ae00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762831.529039:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0448e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529043:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0448e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529049:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26d00) 00010000:00010000:2.0:1789762831.529051:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762831.529058:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0448e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529062:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0448e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529070:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b400/0x416193e012b04495 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529074:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b400/0x416193e012b04495 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529078:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b400/0x416193e012b04495 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529082:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762831.529103:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b400/0x416193e012b04495 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529108:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b400/0x416193e012b04495 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529116:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e82a26c00) 00010000:00010000:2.0:1789762831.529118:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762831.529120:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b400/0x416193e012b04495 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529139:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1b400/0x416193e012b04495 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529162:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a000/0x416193e012b04487 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd68d expref: 30 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529182:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26d00) 00010000:00010000:2.0:1789762831.529183:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762831.529187:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a000/0x416193e012b04487 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd68d expref: 30 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.529197:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87c1a000, rc 0) 00000100:00100000:2.0:1789762831.529226:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:14291:x1876702015028096:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 513us (610us total) trans 0 rc 301/301 00000100:00100000:2.0:1789762831.529233:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 293 00000100:00100000:2.0:1789762831.530035:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762831.530068:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015028224 00000100:00100000:3.0:1789762831.530074:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015028224 00000100:00100000:3.0:1789762831.530233:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 294 00000100:00100000:3.0:1789762831.530242:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:14291:x1876702015028224:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762831.530250:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762831.530261:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044a3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd694 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.530295:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b044aa lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.530302:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b044aa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.530308:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b044aa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762831.530323:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762831.530325:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762831.530333:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b044aa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.530345:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044a3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd694 expref: 31 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.530355:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b044aa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd694 expref: 30 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.530361:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:3.0:1789762831.530363:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762831.530365:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb4800, rc 0) 00000100:00100000:3.0:1789762831.530393:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14291:x1876702015028224:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 154us (358us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762831.530398:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 294 00000100:00100000:2.0:1789762831.531213:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.531247:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015028480 00000100:00100000:1.0:1789762831.531254:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015028480 00000100:00100000:1.0:1789762831.531283:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 295 00000100:00100000:1.0:1789762831.531289:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:14291:x1876702015028480:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762831.531296:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762831.531305:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b044b1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd69b expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531333:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b044b8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531343:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b044b8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531351:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b044b8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531365:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b044bf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531374:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b044bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531382:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b044bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762831.531435:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00010000:00010000:1.0:1789762831.531479:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b044b8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531484:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b044b8 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531489:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b044b8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531493:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762831.531495:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b044b8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531501:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b044b8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531513:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:1.0:1789762831.531516:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762831.531520:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b044b8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531527:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482e00/0x416193e012b044b8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531541:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b044bf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531549:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b044bf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531560:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b044bf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531564:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762831.531566:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b044bf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531570:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b044bf lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531576:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179f00) 00010000:00010000:1.0:1789762831.531577:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762831.531579:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b044bf lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531583:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81483400/0x416193e012b044bf lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531605:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b044b1 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd69b expref: 32 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531612:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:1.0:1789762831.531614:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789762831.531616:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482000/0x416193e012b044b1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd69b expref: 32 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.531623:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81482000, rc 0) 00000100:00100000:1.0:1789762831.531655:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:14291:x1876702015028480:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 366us (442us total) trans 4294967547 rc 301/301 00000100:00100000:1.0:1789762831.531664:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 295 00000100:00100000:1.0:1789762831.532456:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762831.532483:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015028736 00000100:00100000:2.0:1789762831.532488:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015028736 00000100:00100000:2.0:1789762831.532520:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 296 00000100:00100000:2.0:1789762831.532527:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc556d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:14291:x1876702015028736:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:2.0:1789762831.532558:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ac00/0x416193e012b044cd lrc: 3/0,1 mode: --/PW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.532566:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ac00/0x416193e012b044cd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.532573:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ac00/0x416193e012b044cd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762831.532696:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d850c671 reply state 00000000438b09a9 transno 4294967548 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ac00/0x416193e012b044cd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762831.532719:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762831.532726:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00000100:00100000:2.0:1789762831.532775:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc556d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:14291:x1876702015028736:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 248us (319us total) trans 4294967548 rc 0/0 00000100:00100000:2.0:1789762831.532781:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 296 00010000:00010000:1.0:1789762831.533315:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ac00/0x416193e012b044cd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.533322:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ac00/0x416193e012b044cd lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.533327:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ac00/0x416193e012b044cd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.533333:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ac00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762831.533337:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ac00/0x416193e012b044cd lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.533344:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ac00/0x416193e012b044cd lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.533355:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981e82a26200) 00010000:00010000:1.0:1789762831.533358:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789762831.533362:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ac00/0x416193e012b044cd lrc: 2/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.533370:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ac00/0x416193e012b044cd lrc: 0/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762831.533505:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762831.533539:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015028992 00000100:00100000:2.0:1789762831.533544:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015028992 00000100:00100000:2.0:1789762831.533572:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 93 00000100:00100000:2.0:1789762831.533578:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:14291:x1876702015028992:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:2.0:1789762831.533678:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:14291:x1876702015028992:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 99us (173us total) trans 4294967549 rc 0/0 00000100:00100000:2.0:1789762831.533687:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 93 00000100:00100000:2.0:1789762831.537001:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762831.537032:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015029120 00000100:00100000:0.0:1789762831.537038:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015029120 00000100:00100000:0.0:1789762831.537067:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 297 00000100:00100000:0.0:1789762831.537074:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:14292:x1876702015029120:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:0.0:1789762831.537170:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762831.537182:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b044d4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6a2 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537211:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b044db lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537218:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b044db lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537225:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b044db lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537234:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b044e2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537238:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b044e2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537243:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b044e2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537286:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b044e9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537291:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b044e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537296:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b044e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762831.537308:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00000004:00200000:0.0:1789762831.537316:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762831.537328:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b044e9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537335:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b044db lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537339:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b044db lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537343:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b044db lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537346:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762831.537349:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b044db lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537354:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b044db lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537360:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:0.0:1789762831.537362:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762831.537365:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b044db lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537369:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b044db lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537377:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b044e2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537382:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b044e2 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537394:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b044e2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537397:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbea00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762831.537399:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b044e2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537403:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b044e2 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537411:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3c00) 00010000:00010000:0.0:1789762831.537413:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762831.537415:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b044e2 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537419:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b044e2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537442:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b044d4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6a2 expref: 35 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537453:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b044e9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6a2 expref: 34 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.537458:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981ea0aa3500) 00010000:00010000:0.0:1789762831.537460:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762831.537461:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbe600, rc 0) 00000100:00100000:0.0:1789762831.537490:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:14292:x1876702015029120:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 417us (493us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762831.537504:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 297 00000100:00100000:2.0:1789762831.538279:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762831.538312:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015029376 00000100:00100000:0.0:1789762831.538317:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015029376 00000100:00100000:0.0:1789762831.538341:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 298 00000100:00100000:0.0:1789762831.538347:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:14292:x1876702015029376:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:0.0:1789762831.538367:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762831.538376:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b044e9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6a2 expref: 34 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.538385:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981ea0aa3500) 00010000:00010000:0.0:1789762831.538390:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762831.538400:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b044f0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.538406:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b044f0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.538411:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b044f0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762831.538507:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000017340f80 reply state 00000000d78ff01b transno 4294967550 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b044f0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.538512:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762831.538516:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00000100:00100000:0.0:1789762831.538551:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:14292:x1876702015029376:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 204us (274us total) trans 4294967550 rc 0/0 00000100:00100000:0.0:1789762831.538556:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 298 00010000:00010000:0.0:1789762831.538949:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b044f0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.538955:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b044f0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.538959:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b044f0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.538963:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762831.538965:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b044f0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.538969:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b044f0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.538976:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981ea0aa3e00) 00010000:00010000:0.0:1789762831.538978:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789762831.538980:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b044f0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.538984:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbf000/0x416193e012b044f0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762831.557634:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762831.557701:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015029504 00000100:00100000:3.0:1789762831.557710:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015029504 00000100:00100000:3.0:1789762831.557756:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 299 00000100:00100000:3.0:1789762831.557764:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:14298:x1876702015029504:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762831.557782:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762831.557797:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b044f7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6a9 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.557842:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044fe lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.557852:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044fe lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.557860:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044fe lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.557869:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04505 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.557874:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04505 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.557878:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04505 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.557953:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b0450c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.557958:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b0450c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.557963:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b0450c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762831.557987:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00000004:00200000:3.0:1789762831.557990:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762831.558003:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b0450c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558011:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044fe lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558016:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044fe lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558020:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044fe lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558024:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762831.558027:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044fe lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558032:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044fe lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558041:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:3.0:1789762831.558043:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762831.558046:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044fe lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558050:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b044fe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558057:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04505 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558068:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04505 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558072:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04505 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558075:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762831.558077:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04505 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558081:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04505 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558110:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272c300) 00010000:00010000:3.0:1789762831.558113:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762831.558116:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04505 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558120:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04505 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558166:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b044f7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6a9 expref: 36 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558179:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b0450c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6a9 expref: 35 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.558185:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981e9272c600) 00010000:00010000:3.0:1789762831.558187:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789762831.558194:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb4000, rc 0) 00000100:00100000:3.0:1789762831.558237:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:14298:x1876702015029504:12345-192.168.203.48@tcp:101:dd.500 Request processed in 474us (609us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762831.558244:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 299 00000100:00100000:3.0:1789762831.559049:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762831.559158:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015029760 00000100:00100000:0.0:1789762831.559164:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015029760 00000100:00100000:0.0:1789762831.559198:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 300 00000100:00100000:0.0:1789762831.559207:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:14298:x1876702015029760:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762831.559215:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762831.559224:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b04513 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6b0 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559299:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0451a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559307:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0451a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559313:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0451a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559321:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b04521 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559326:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b04521 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559330:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b04521 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762831.559659:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x14:0x0], index 0: 0 00010000:00010000:0.0:1789762831.559768:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04528 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559775:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04528 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559780:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04528 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559799:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0451a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559811:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0451a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559815:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0451a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559819:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762831.559821:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0451a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559825:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0451a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559831:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981e9272c600) 00010000:00010000:0.0:1789762831.559833:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762831.559836:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0451a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559840:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe000/0x416193e012b0451a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762831.559847:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f95f8162 reply state 000000004629c845 transno 4294967551 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b04521 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762831.559853:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762831.559869:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe600/0x416193e012b04513 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6b0 expref: 38 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559877:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04528 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6b0 expref: 37 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.559890:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff981ea0aa3500) 00010000:00010000:0.0:1789762831.559892:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762831.559893:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87cbe200, rc 0) 00000100:00100000:0.0:1789762831.559924:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:14298:x1876702015029760:12345-192.168.203.48@tcp:101:dd.500 Request processed in 720us (877us total) trans 4294967551 rc 0/0 00000100:00100000:0.0:1789762831.559929:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 300 00010000:00010000:1.0:1789762831.560451:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b04521 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.560460:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b04521 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.560465:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b04521 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.560469:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbea00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762831.560471:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b04521 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.560475:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b04521 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.560482:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x4009f790 (ffff981ea0aa3e00) 00010000:00010000:1.0:1789762831.560484:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762831.560487:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b04521 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.560490:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbea00/0x416193e012b04521 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762831.562817:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762831.562882:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015029888 00000100:00100000:0.0:1789762831.562888:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015029888 00000100:00100000:0.0:1789762831.562921:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 354394771489 time 354394843948 state 0x0 ref 1 00000100:00100000:0.0:1789762831.562931:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 354494843948 time 354394843948 state 0x1 ref 1 00000100:00100000:0.0:1789762831.562955:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 354494843948 time 354866864097 state 0x0 ref 1 00000100:00100000:0.0:1789762831.562963:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2366:x1876702015029888:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762831.562991:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789762831.562996:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04536 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.563007:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=354s v=(0 0) 00010000:00010000:0.0:1789762831.563011:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.563015:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04536 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.563034:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04536 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762831.591629:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967929 is committed 00000001:00080000:0.0:1789762831.591666:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967912 is committed 00002000:00010000:1.0:1789762831.591672:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:1.0:1789762831.591677:0:14914:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 0 -> 1789762831 00002000:00010000:1.0:1789762831.591679:0:14914:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 0 -> 1789762831 00002000:00010000:1.0:1789762831.591681:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:1.0:1789762831.591694:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04536 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.591705:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04536 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.591712:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04536 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.591719:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762831.591722:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04536 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.591729:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04536 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.591743:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04536 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.591748:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b112442/0x416193e012b04536 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762831.591838:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015029888:12345-192.168.203.48@tcp:4:dd.500 Request processed in 28870us (29017us total) trans 4294967929 rc 0/0 00000100:00100000:1.0:1789762831.591847:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 726 00000100:00100000:1.0:1789762831.591857:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 354494843948 time 354866864097 state 0x2 ref 0 00000100:00100000:3.0:1789762831.606142:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.606202:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015030016 00000100:00100000:1.0:1789762831.606211:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015030016 00000100:00100000:1.0:1789762831.606272:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 354494843948 time 354866864097 state 0x0 ref 1 00000100:00100000:1.0:1789762831.606285:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 354966864097 time 354866864097 state 0x1 ref 1 00000100:00100000:1.0:1789762831.606311:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 354966864097 time 354910220284 state 0x0 ref 1 00000100:00100000:1.0:1789762831.606323:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015030016:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762831.606367:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 1048576, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x803 00010000:00010000:1.0:1789762831.606377:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0453d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.606392:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=354s v=(0 0) 00010000:00010000:1.0:1789762831.606396:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.606401:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0453d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.606412:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0453d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762831.620808:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967930 is committed 00002000:00010000:1.0:1789762831.620832:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:1.0:1789762831.620837:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:1.0:1789762831.620849:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0453d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.620859:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0453d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.620865:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0453d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.620870:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762831.620874:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0453d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.620882:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0453d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.620895:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0453d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.620900:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0453d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762831.620965:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015030016:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14642us (14841us total) trans 4294967930 rc 0/0 00000100:00100000:1.0:1789762831.620973:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 727 00000100:00100000:1.0:1789762831.620982:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 354966864097 time 354910220284 state 0x2 ref 0 00000100:00100000:3.0:1789762831.623571:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762831.623620:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015030144 00000100:00100000:0.0:1789762831.623628:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015030144 00000100:00100000:0.0:1789762831.623667:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 354966864097 time 354910220284 state 0x0 ref 1 00000100:00100000:0.0:1789762831.623678:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 355010220284 time 354910220284 state 0x1 ref 1 00000100:00100000:0.0:1789762831.623700:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355010220284 time 354927608709 state 0x0 ref 1 00000100:00100000:0.0:1789762831.623709:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015030144:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762831.623736:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 2097152, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x1007 00010000:00010000:0.0:1789762831.623741:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04544 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.623751:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=354s v=(0 0) 00010000:00010000:0.0:1789762831.623754:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.623759:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04544 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.623769:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04544 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762831.636905:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967931 is committed 00002000:00010000:3.0:1789762831.636909:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:3.0:1789762831.636914:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:3.0:1789762831.636923:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04544 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.636933:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04544 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.636940:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04544 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.636945:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762831.636948:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04544 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.636954:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04544 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.636965:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04544 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.636980:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04544 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762831.637031:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015030144:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13321us (13459us total) trans 4294967931 rc 0/0 00000100:00100000:3.0:1789762831.637038:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 728 00000100:00100000:3.0:1789762831.637046:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355010220284 time 354927608709 state 0x2 ref 0 00000100:00100000:0.0:1789762831.639323:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.639389:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015030272 00000100:00100000:1.0:1789762831.639395:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015030272 00000100:00100000:1.0:1789762831.639426:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355010220284 time 354927608709 state 0x0 ref 1 00000100:00100000:1.0:1789762831.639436:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355027608709 time 354927608709 state 0x1 ref 1 00000100:00100000:1.0:1789762831.723763:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355027608709 time 355027659793 state 0x0 ref 1 00000100:00100000:1.0:1789762831.723787:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015030272:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762831.723835:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 3145728, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x1807 00010000:00010000:1.0:1789762831.723843:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0454b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.723856:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=355s v=(0 0) 00010000:00010000:1.0:1789762831.723860:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.723865:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0454b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.723877:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0454b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762831.736786:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967932 is committed 00002000:00010000:2.0:1789762831.736806:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:2.0:1789762831.736813:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:2.0:1789762831.736826:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0454b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.736839:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0454b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.736850:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0454b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.736874:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762831.736879:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0454b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.736890:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0454b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.736905:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0454b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.736911:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0454b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762831.736970:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015030272:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13190us (97645us total) trans 4294967932 rc 0/0 00000100:00100000:2.0:1789762831.736978:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 729 00000100:00100000:2.0:1789762831.736987:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355027608709 time 355027659793 state 0x2 ref 0 00000100:00100000:2.0:1789762831.739387:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762831.739443:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015030400 00000100:00100000:0.0:1789762831.739454:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015030400 00000100:00100000:0.0:1789762831.739503:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355027608709 time 355027659793 state 0x0 ref 1 00000100:00100000:0.0:1789762831.739516:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355127659793 time 355027659793 state 0x1 ref 1 00000100:00100000:0.0:1789762831.823836:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355127659793 time 355127728344 state 0x0 ref 1 00000100:00100000:0.0:1789762831.823866:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015030400:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762831.823924:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 4194304, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2007 00010000:00010000:0.0:1789762831.823933:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b04552 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.823949:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=355s v=(0 0) 00010000:00010000:0.0:1789762831.823954:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.823960:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b04552 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.823975:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b04552 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762831.837955:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967933 is committed 00002000:00010000:2.0:1789762831.837984:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:2.0:1789762831.837989:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:2.0:1789762831.838002:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b04552 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.838012:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b04552 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.838027:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b04552 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.838032:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762831.838036:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b04552 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.838042:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b04552 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.838059:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b04552 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.838064:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b04552 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762831.838178:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015030400:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14319us (98786us total) trans 4294967933 rc 0/0 00000100:00100000:2.0:1789762831.838187:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 730 00000100:00100000:2.0:1789762831.838197:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355127659793 time 355127728344 state 0x2 ref 0 00000100:00100000:0.0:1789762831.840879:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.840933:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015030528 00000100:00100000:1.0:1789762831.840942:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015030528 00000100:00100000:1.0:1789762831.840982:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355127659793 time 355127728344 state 0x0 ref 1 00000100:00100000:1.0:1789762831.840993:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355227728344 time 355127728344 state 0x1 ref 1 00000100:00100000:3.0:1789762831.923915:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355227728344 time 355227812139 state 0x0 ref 1 00000100:00100000:3.0:1789762831.923936:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015030528:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762831.923983:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 5242880, mtime: 0x6aad9d0f, atime: 0x0, ctime: 0x6aad9d0f, blocks: 0x2807 00010000:00010000:3.0:1789762831.923990:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b04559 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.924004:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=355s v=(0 0) 00010000:00010000:3.0:1789762831.924007:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762831.924012:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b04559 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762831.924023:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b04559 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762831.937780:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967934 is committed 00002000:00010000:2.0:1789762831.937844:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:2.0:1789762831.937849:0:14912:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 1789762831 -> 1789762832 00002000:00010000:2.0:1789762831.937851:0:14912:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 1789762831 -> 1789762832 00002000:00010000:2.0:1789762831.937853:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:2.0:1789762831.937869:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b04559 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.937878:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b04559 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.937884:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b04559 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.937889:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762831.937893:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b04559 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.937900:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b04559 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.937913:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b04559 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762831.937918:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000033fe766/0x416193e012b04559 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762831.937976:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902ded80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015030528:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14046us (97097us total) trans 4294967934 rc 0/0 00000100:00100000:2.0:1789762831.937984:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 731 00000100:00100000:2.0:1789762831.937993:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355227728344 time 355227812139 state 0x2 ref 0 00000100:00100000:2.0:1789762831.940461:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762831.940527:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015030656 00000100:00100000:1.0:1789762831.940535:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015030656 00000100:00100000:1.0:1789762831.940572:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355227728344 time 355227812139 state 0x0 ref 1 00000100:00100000:1.0:1789762831.940584:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355327812139 time 355227812139 state 0x1 ref 1 00000001:00080000:1.0:1789762831.999519:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967551 is committed 00000004:00080000:1.0:1789762831.999569:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 19 committed 00000004:00080000:1.0:1789762831.999584:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 23 committed 00000100:00100000:3.0:1789762831.999711:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac6ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022171776:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1789762831.999766:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762831.999793:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022171776 00000100:00100000:2.0:1789762831.999800:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022171776 00000100:00100000:3.0:1789762831.999813:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e902dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022171648:0@lo:6:osp-syn-1-0.0 00000100:00080000:1.0:1789762831.999830:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762831.999840:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762831.999852:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022171648 00000100:00100000:2.0:1789762831.999856:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022171648 00000100:00100000:1.0:1789762831.999869:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:1.0:1789762831.999883:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022171776:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789762831.999888:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:0.0:1789762831.999900:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022171648:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1789762831.999909:0:6881:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:18 count 1 00002000:00080000:0.0:1789762831.999917:0:10916:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:12 count 1 00010000:00010000:1.0:1789762831.999931:0:6881:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762831.999940:0:10916:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xc:0x0] initial LVB size: 8388608, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x803 00010000:00010000:0.0:1789762831.999948:0:10916:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009379584a/0x416193e012b04567 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762831.999951:0:6881:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b043d8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd663 expref: 6 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.999973:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=355s v=(0 0) 00010000:00010000:1.0:1789762831.999973:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762831.999978:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762831.999979:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762831.999983:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762831.999985:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009379584a/0x416193e012b04567 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.000006:0:10916:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009379584a/0x416193e012b04567 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.000013:0:10916:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000009379584a/0x416193e012b04567 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.000017:0:6881:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b043d8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd663 expref: 6 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.000028:0:10916:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009379584a/0x416193e012b04567 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.000036:0:6881:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b043d8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd663 expref: 6 pid: 6649 timeout: 455 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762832.000047:0:6881:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81394000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:6881:1876702022171904:192.168.203.48@tcp:104: 00010000:00010000:0.0:1789762832.000048:0:10916:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762832.000057:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762832.000065:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009379584a/0x416193e012b04567 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.000073:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762832.000077:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009379584a/0x416193e012b04567 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.000099:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009379584a/0x416193e012b04567 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.000115:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009379584a/0x416193e012b04567 lrc: 1/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.000132:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009379584a/0x416193e012b04567 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.000157:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762832.000160:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762832.000162:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762832.000168:0:6881:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e90992540 going to sleep for 10 seconds 00000100:00080000:1.0:1789762832.000270:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762832.000499:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022171648:12345-0@lo:6:osp-syn-1-0.0 Request processed in 601us (659us total) trans 4294967415 rc 0/0 00000100:00100000:0.0:1789762832.000505:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000004:00080000:3.0:1789762832.000515:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981e902dd500/2, rc 0, transno 4294967415 00000100:00100000:3.0:1789762832.000531:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e902dd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022171648:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1789762832.001247:0:6881:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81394000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:6881:1876702022171904:192.168.203.48@tcp:104: 00010000:00010000:1.0:1789762832.001265:0:6881:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762832.001352:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762832.001392:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015030784 00000100:00100000:0.0:1789762832.001398:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015030784 00010000:00010000:0.0:1789762832.001435:0:5791:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b043d8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd663 expref: 7 pid: 6649 timeout: 455 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762832.001462:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 61 00000100:00080000:1.0:1789762832.001465:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762832.001471:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2363:x1876702015030784:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762832.001480:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762832.001485:0:5791:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b043d8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd663 expref: 7 pid: 6649 timeout: 455 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.001495:0:5791:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b043d8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd663 expref: 7 pid: 6649 timeout: 455 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.001508:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b043d8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd663 expref: 7 pid: 6649 timeout: 455 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.001516:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762832.001519:0:5791:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762832.001523:0:5791:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.001529:0:5791:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762832.001543:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762832.001562:0:6881:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762832.001570:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015030784:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 101us (219us total) trans 0 rc 0/0 00010000:00010000:1.0:1789762832.001585:0:6881:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762832.001586:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 61 00010000:00010000:1.0:1789762832.001601:0:6881:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.001611:0:6881:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.001632:0:6881:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762832.001673:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762832.001680:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.001688:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762832.001691:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.001698:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.001707:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.001712:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04560 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762832.001723:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762832.001730:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762832.001732:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762832.001996:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022171776:12345-0@lo:6:osp-syn-0-0.0 Request processed in 2117us (2228us total) trans 4294967935 rc 0/0 00000100:00100000:1.0:1789762832.002003:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:3.0:1789762832.002018:0:3271:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967912 gen 2 00000100:00100000:3.0:1789762832.002032:0:3271:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff981ebac6ed80 x1876702022171776/t4294967935(4294967935) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789762847 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:3.0:1789762832.002039:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebac6ed80/2, rc 0, transno 4294967935 00000100:00100000:3.0:1789762832.002049:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac6ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022171776:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762832.024000:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355327812139 time 355327899818 state 0x0 ref 1 00000100:00100000:2.0:1789762832.024024:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015030656:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762832.024055:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 6291456, mtime: 0x6aad9d10, atime: 0x0, ctime: 0x6aad9d10, blocks: 0x3007 00010000:00010000:2.0:1789762832.024061:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0456e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.024071:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=355s v=(0 0) 00010000:00010000:2.0:1789762832.024074:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762832.024078:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0456e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.024106:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0456e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762832.037690:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967935 is committed 00000001:00080000:1.0:1789762832.037702:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967936 is committed 00002000:00010000:2.0:1789762832.037705:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:2.0:1789762832.037711:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:2.0:1789762832.037722:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0456e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.037737:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0456e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.037748:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0456e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.037758:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762832.037770:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0456e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.037781:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0456e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.037799:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0456e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.037808:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0456e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762832.037872:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015030656:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13848us (97410us total) trans 4294967936 rc 0/0 00000100:00100000:2.0:1789762832.037881:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 732 00000100:00100000:2.0:1789762832.037889:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355327812139 time 355327899818 state 0x2 ref 0 00000100:00100000:2.0:1789762832.040340:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762832.040411:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015030912 00000100:00100000:0.0:1789762832.040421:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015030912 00000100:00100000:0.0:1789762832.040465:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355327812139 time 355327899818 state 0x0 ref 1 00000100:00100000:0.0:1789762832.040481:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355427899818 time 355327899818 state 0x1 ref 1 00000100:00100000:2.0:1789762832.124173:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355427899818 time 355428067620 state 0x0 ref 1 00000100:00100000:2.0:1789762832.124199:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015030912:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762832.124248:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 7340032, mtime: 0x6aad9d10, atime: 0x0, ctime: 0x6aad9d10, blocks: 0x3807 00010000:00010000:2.0:1789762832.124255:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04575 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.124269:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=355s v=(0 0) 00010000:00010000:2.0:1789762832.124285:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762832.124290:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04575 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.124301:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04575 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762832.137722:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967937 is committed 00002000:00010000:2.0:1789762832.137742:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:2.0:1789762832.137749:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:2.0:1789762832.137763:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04575 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.137778:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04575 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.137788:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04575 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.137797:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762832.137802:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04575 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.137813:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04575 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.137833:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04575 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.137855:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04575 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762832.137934:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015030912:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13741us (97590us total) trans 4294967937 rc 0/0 00000100:00100000:2.0:1789762832.137943:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 733 00000100:00100000:2.0:1789762832.137953:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355427899818 time 355428067620 state 0x2 ref 0 00000100:00100000:2.0:1789762832.140353:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762832.140409:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015031040 00000100:00100000:1.0:1789762832.140416:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015031040 00000100:00100000:1.0:1789762832.140455:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355427899818 time 355428067620 state 0x0 ref 1 00000100:00100000:1.0:1789762832.140469:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355528067620 time 355428067620 state 0x1 ref 1 00000100:00100000:2.0:1789762832.224269:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355528067620 time 355528164881 state 0x0 ref 1 00000100:00100000:2.0:1789762832.224293:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015031040:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762832.224343:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 8388608, mtime: 0x6aad9d10, atime: 0x0, ctime: 0x6aad9d10, blocks: 0x4007 00010000:00010000:2.0:1789762832.224351:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0457c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.224364:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=355s v=(0 0) 00010000:00010000:2.0:1789762832.224368:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762832.224372:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0457c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.224384:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0457c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762832.236716:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967938 is committed 00002000:00010000:2.0:1789762832.236744:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:2.0:1789762832.236751:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:2.0:1789762832.236763:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0457c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.236777:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0457c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.236788:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0457c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.236797:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762832.236802:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0457c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.236812:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0457c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.236830:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0457c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.236835:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0457c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762832.236895:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015031040:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12609us (96541us total) trans 4294967938 rc 0/0 00000100:00100000:2.0:1789762832.236910:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 734 00000100:00100000:2.0:1789762832.236919:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355528067620 time 355528164881 state 0x2 ref 0 00000100:00100000:1.0:1789762832.239258:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762832.239301:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015031168 00000100:00100000:2.0:1789762832.239309:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015031168 00000100:00100000:2.0:1789762832.239351:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355528067620 time 355528164881 state 0x0 ref 1 00000100:00100000:2.0:1789762832.239362:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355628164881 time 355528164881 state 0x1 ref 1 00010000:00010000:1.0:1789762832.311231:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762832.324367:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355628164881 time 355628259156 state 0x0 ref 1 00000100:00100000:2.0:1789762832.324395:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015031168:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762832.324450:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 9437184, mtime: 0x6aad9d10, atime: 0x0, ctime: 0x6aad9d10, blocks: 0x4807 00010000:00010000:2.0:1789762832.324461:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04583 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.324480:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=355s v=(0 0) 00010000:00010000:2.0:1789762832.324484:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762832.324488:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04583 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.324500:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04583 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762832.336498:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967939 is committed 00002000:00010000:2.0:1789762832.336522:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:2.0:1789762832.336535:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 18439 -> 20487 00010000:00010000:2.0:1789762832.336546:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04583 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.336556:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04583 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.336562:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04583 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.336567:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762832.336571:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04583 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.336577:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04583 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.336592:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04583 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.336597:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04583 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762832.336662:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015031168:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12275us (97403us total) trans 4294967939 rc 0/0 00000100:00100000:2.0:1789762832.336671:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 735 00000100:00100000:2.0:1789762832.336681:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355628164881 time 355628259156 state 0x2 ref 0 00000100:00100000:3.0:1789762832.339191:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762832.339244:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015031296 00000100:00100000:0.0:1789762832.339252:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015031296 00000100:00100000:0.0:1789762832.339290:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355628164881 time 355628259156 state 0x0 ref 1 00000100:00100000:0.0:1789762832.339301:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355728259156 time 355628259156 state 0x1 ref 1 00000100:00100000:2.0:1789762832.424445:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355728259156 time 355728342516 state 0x0 ref 1 00000100:00100000:2.0:1789762832.424469:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015031296:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762832.424517:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 10485760, mtime: 0x6aad9d10, atime: 0x0, ctime: 0x6aad9d10, blocks: 0x5007 00010000:00010000:2.0:1789762832.424524:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0458a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.424537:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=355s v=(0 0) 00010000:00010000:2.0:1789762832.424540:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762832.424545:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0458a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.424556:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0458a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762832.435802:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967940 is committed 00002000:00010000:2.0:1789762832.435824:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:2.0:1789762832.435830:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:2.0:1789762832.435842:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0458a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.435855:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0458a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.435865:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0458a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.435887:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762832.435893:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0458a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.435903:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0458a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.435922:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0458a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.435929:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b0458a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762832.435990:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015031296:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11528us (96800us total) trans 4294967940 rc 0/0 00000100:00100000:2.0:1789762832.435999:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 736 00000100:00100000:2.0:1789762832.436008:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355728259156 time 355728342516 state 0x2 ref 0 00000100:00100000:3.0:1789762832.438590:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762832.438665:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015031424 00000100:00100000:1.0:1789762832.438674:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015031424 00000100:00100000:1.0:1789762832.438712:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355728259156 time 355728342516 state 0x0 ref 1 00000100:00100000:1.0:1789762832.438723:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355828342516 time 355728342516 state 0x1 ref 1 00000100:00100000:2.0:1789762832.524564:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355828342516 time 355828457261 state 0x0 ref 1 00000100:00100000:2.0:1789762832.524592:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015031424:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762832.524647:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 11534336, mtime: 0x6aad9d10, atime: 0x0, ctime: 0x6aad9d10, blocks: 0x5807 00010000:00010000:2.0:1789762832.524656:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04591 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.524684:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=355s v=(0 0) 00010000:00010000:2.0:1789762832.524688:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762832.524694:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04591 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.524708:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04591 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762832.537319:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 11534336 -> 12582912 00000001:00080000:3.0:1789762832.537324:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967941 is committed 00002000:00010000:2.0:1789762832.537326:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 22535 -> 24583 00010000:00010000:2.0:1789762832.537342:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04591 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.537356:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04591 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.537368:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04591 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.537376:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762832.537380:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04591 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.537387:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04591 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.537413:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04591 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.537419:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04591 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762832.537489:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015031424:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12904us (98899us total) trans 4294967941 rc 0/0 00000100:00100000:2.0:1789762832.537499:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 737 00000100:00100000:2.0:1789762832.537511:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355828342516 time 355828457261 state 0x2 ref 0 00000100:00100000:3.0:1789762832.539756:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762832.539824:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015031552 00000100:00100000:2.0:1789762832.539833:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015031552 00000100:00100000:2.0:1789762832.539870:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355828342516 time 355828457261 state 0x0 ref 1 00000100:00100000:2.0:1789762832.539881:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355928457261 time 355828457261 state 0x1 ref 1 00000100:00100000:2.0:1789762832.624629:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355928457261 time 355928525281 state 0x0 ref 1 00000100:00100000:2.0:1789762832.624653:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015031552:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762832.624701:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 12582912, mtime: 0x6aad9d10, atime: 0x0, ctime: 0x6aad9d10, blocks: 0x6007 00010000:00010000:2.0:1789762832.624708:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04598 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.624740:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=355s v=(0 0) 00010000:00010000:2.0:1789762832.624743:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762832.624748:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04598 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.624760:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04598 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762832.636607:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967942 is committed 00002000:00010000:1.0:1789762832.636638:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 12582912 -> 13631488 00002000:00010000:1.0:1789762832.636645:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:1.0:1789762832.636656:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04598 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.636669:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04598 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.636678:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04598 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.636686:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762832.636692:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04598 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.636708:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04598 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.636727:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04598 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.636736:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b04598 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762832.636807:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015031552:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12159us (97048us total) trans 4294967942 rc 0/0 00000100:00100000:1.0:1789762832.636817:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 738 00000100:00100000:1.0:1789762832.636829:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355928457261 time 355928525281 state 0x2 ref 0 00000100:00100000:3.0:1789762832.639008:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762832.639102:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015031680 00000100:00100000:0.0:1789762832.639111:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015031680 00000100:00100000:0.0:1789762832.639165:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 355928457261 time 355928525281 state 0x0 ref 1 00000100:00100000:0.0:1789762832.639176:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356028525281 time 355928525281 state 0x1 ref 1 00000100:00100000:2.0:1789762832.724720:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356028525281 time 356028612565 state 0x0 ref 1 00000100:00100000:2.0:1789762832.724748:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015031680:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762832.724806:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 13631488, mtime: 0x6aad9d10, atime: 0x0, ctime: 0x6aad9d10, blocks: 0x6807 00010000:00010000:2.0:1789762832.724816:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b0459f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.724831:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=356s v=(0 0) 00010000:00010000:2.0:1789762832.724837:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762832.724844:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b0459f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.724862:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b0459f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762832.737366:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967943 is committed 00002000:00010000:2.0:1789762832.737444:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:2.0:1789762832.737455:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:2.0:1789762832.737467:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b0459f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.737479:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b0459f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.737489:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b0459f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.737497:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762832.737503:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b0459f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.737513:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b0459f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.737532:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b0459f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.737541:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b0459f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762832.737614:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015031680:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12873us (98605us total) trans 4294967943 rc 0/0 00000100:00100000:2.0:1789762832.737626:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 739 00000100:00100000:2.0:1789762832.737638:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356028525281 time 356028612565 state 0x2 ref 0 00000100:00100000:3.0:1789762832.739820:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762832.739862:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015031808 00000100:00100000:1.0:1789762832.739870:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015031808 00000100:00100000:1.0:1789762832.739906:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356028525281 time 356028612565 state 0x0 ref 1 00000100:00100000:1.0:1789762832.739917:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356128612565 time 356028612565 state 0x1 ref 1 00000100:00100000:2.0:1789762832.824811:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356128612565 time 356128700916 state 0x0 ref 1 00000100:00100000:2.0:1789762832.824840:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4970a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015031808:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762832.824895:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 14680064, mtime: 0x6aad9d10, atime: 0x0, ctime: 0x6aad9d10, blocks: 0x7007 00010000:00010000:2.0:1789762832.824905:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b045a6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.824921:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=356s v=(0 0) 00010000:00010000:2.0:1789762832.824927:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762832.824934:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b045a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.824949:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b045a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762832.837028:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967944 is committed 00002000:00010000:2.0:1789762832.837056:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:2.0:1789762832.837062:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:2.0:1789762832.837074:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b045a6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.837104:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b045a6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.837128:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b045a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.837152:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762832.837157:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b045a6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.837167:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b045a6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.837186:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b045a6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.837195:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a90c158b/0x416193e012b045a6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762832.837267:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4970a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015031808:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12435us (97445us total) trans 4294967944 rc 0/0 00000100:00100000:2.0:1789762832.837278:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 740 00000100:00100000:2.0:1789762832.837291:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356128612565 time 356128700916 state 0x2 ref 0 00000100:00100000:3.0:1789762832.839462:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762832.839507:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015031936 00000100:00100000:2.0:1789762832.839517:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015031936 00000100:00100000:2.0:1789762832.839561:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356128612565 time 356128700916 state 0x0 ref 1 00000100:00100000:2.0:1789762832.839579:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356228700916 time 356128700916 state 0x1 ref 1 00000100:00100000:1.0:1789762832.924893:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356228700916 time 356228789185 state 0x0 ref 1 00000100:00100000:1.0:1789762832.924916:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4972d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015031936:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762832.924965:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 15728640, mtime: 0x6aad9d10, atime: 0x0, ctime: 0x6aad9d10, blocks: 0x7807 00010000:00010000:1.0:1789762832.924972:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b045ad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.924985:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=356s v=(0 0) 00010000:00010000:1.0:1789762832.924989:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762832.924994:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b045ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762832.925004:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b045ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762832.936690:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967945 is committed 00002000:00010000:2.0:1789762832.936715:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:2.0:1789762832.936727:0:6654:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 1789762832 -> 1789762833 00002000:00010000:2.0:1789762832.936730:0:6654:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 1789762832 -> 1789762833 00002000:00010000:2.0:1789762832.936733:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:2.0:1789762832.936745:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b045ad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.936757:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b045ad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.936766:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b045ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.936775:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762832.936780:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b045ad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.936791:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b045ad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.936810:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b045ad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762832.936819:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b045ad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762832.936890:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4972d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015031936:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11979us (97426us total) trans 4294967945 rc 0/0 00000100:00100000:2.0:1789762832.936911:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 741 00000100:00100000:2.0:1789762832.936924:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356228700916 time 356228789185 state 0x2 ref 0 00000100:00100000:2.0:1789762832.939106:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762832.939166:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015032064 00000100:00100000:0.0:1789762832.939174:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015032064 00000100:00100000:0.0:1789762832.939212:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356228700916 time 356228789185 state 0x0 ref 1 00000100:00100000:0.0:1789762832.939223:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356328789185 time 356228789185 state 0x1 ref 1 00000100:00100000:2.0:1789762833.024979:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356328789185 time 356328874933 state 0x0 ref 1 00000100:00100000:2.0:1789762833.025003:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015032064:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762833.025054:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 16777216, mtime: 0x6aad9d11, atime: 0x0, ctime: 0x6aad9d11, blocks: 0x8007 00010000:00010000:2.0:1789762833.025061:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009dd2a45c/0x416193e012b045b4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.025075:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=356s v=(0 0) 00010000:00010000:2.0:1789762833.025079:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762833.025084:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009dd2a45c/0x416193e012b045b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.025128:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009dd2a45c/0x416193e012b045b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762833.036880:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 16777216 -> 17825792 00000001:00080000:1.0:1789762833.036888:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967946 is committed 00002000:00010000:3.0:1789762833.036888:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:3.0:1789762833.036904:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009dd2a45c/0x416193e012b045b4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.036919:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009dd2a45c/0x416193e012b045b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.036944:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009dd2a45c/0x416193e012b045b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.036952:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1be00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762833.036957:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009dd2a45c/0x416193e012b045b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.036968:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009dd2a45c/0x416193e012b045b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.036987:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009dd2a45c/0x416193e012b045b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.036996:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009dd2a45c/0x416193e012b045b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762833.037059:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015032064:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12063us (97971us total) trans 4294967946 rc 0/0 00000100:00100000:3.0:1789762833.037067:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 742 00000100:00100000:3.0:1789762833.037076:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356328789185 time 356328874933 state 0x2 ref 0 00000100:00100000:3.0:1789762833.039275:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762833.039316:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015032192 00000100:00100000:1.0:1789762833.039325:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015032192 00000100:00100000:1.0:1789762833.039362:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356328789185 time 356328874933 state 0x0 ref 1 00000100:00100000:1.0:1789762833.039373:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356428874933 time 356328874933 state 0x1 ref 1 00000100:00100000:2.0:1789762833.125124:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356428874933 time 356428983265 state 0x0 ref 1 00000100:00100000:2.0:1789762833.125150:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015032192:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762833.125203:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 17825792, mtime: 0x6aad9d11, atime: 0x0, ctime: 0x6aad9d11, blocks: 0x8807 00010000:00010000:2.0:1789762833.125224:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045bb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.125240:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=356s v=(0 0) 00010000:00010000:2.0:1789762833.125244:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762833.125251:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.125266:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762833.138308:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967947 is committed 00002000:00010000:2.0:1789762833.138335:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:2.0:1789762833.138341:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:2.0:1789762833.138352:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045bb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.138364:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045bb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.138374:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.138382:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762833.138387:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045bb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.138396:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045bb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.138422:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045bb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.138431:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045bb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762833.138502:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015032192:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13357us (99224us total) trans 4294967947 rc 0/0 00000100:00100000:2.0:1789762833.138514:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 743 00000100:00100000:2.0:1789762833.138526:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356428874933 time 356428983265 state 0x2 ref 0 00000100:00100000:3.0:1789762833.140772:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762833.140820:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015032320 00000100:00100000:2.0:1789762833.140828:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015032320 00000100:00100000:2.0:1789762833.140871:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356428874933 time 356428983265 state 0x0 ref 1 00000100:00100000:2.0:1789762833.140885:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356528983265 time 356428983265 state 0x1 ref 1 00000100:00100000:2.0:1789762833.225184:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356528983265 time 356529079278 state 0x0 ref 1 00000100:00100000:2.0:1789762833.225211:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4970000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015032320:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762833.225261:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 18874368, mtime: 0x6aad9d11, atime: 0x0, ctime: 0x6aad9d11, blocks: 0x9007 00010000:00010000:2.0:1789762833.225268:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045c2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.225298:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=356s v=(0 0) 00010000:00010000:2.0:1789762833.225302:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762833.225306:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.225318:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762833.237286:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967948 is committed 00002000:00010000:2.0:1789762833.237309:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:2.0:1789762833.237314:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:2.0:1789762833.237325:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045c2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.237335:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045c2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.237341:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.237346:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762833.237350:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045c2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.237357:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045c2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.237380:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045c2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.237386:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b045c2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762833.237453:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4970000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015032320:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12249us (96679us total) trans 4294967948 rc 0/0 00000100:00100000:2.0:1789762833.237464:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 744 00000100:00100000:2.0:1789762833.237473:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356528983265 time 356529079278 state 0x2 ref 0 00000100:00100000:0.0:1789762833.239625:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762833.239668:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015032448 00000100:00100000:2.0:1789762833.239676:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015032448 00000100:00100000:2.0:1789762833.239711:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356528983265 time 356529079278 state 0x0 ref 1 00000100:00100000:2.0:1789762833.239723:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356629079278 time 356529079278 state 0x1 ref 1 00000100:00100000:3.0:1789762833.325299:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356629079278 time 356629190795 state 0x0 ref 1 00000100:00100000:3.0:1789762833.325340:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015032448:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762833.325396:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 19922944, mtime: 0x6aad9d11, atime: 0x0, ctime: 0x6aad9d11, blocks: 0x9807 00010000:00010000:3.0:1789762833.325407:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b045c9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.325423:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=356s v=(0 0) 00010000:00010000:3.0:1789762833.325428:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762833.325435:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b045c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.325449:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b045c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.335184:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789762833.337585:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967949 is committed 00002000:00010000:1.0:1789762833.337604:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 19922944 -> 20971520 00002000:00010000:1.0:1789762833.337611:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:1.0:1789762833.337623:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b045c9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.337633:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b045c9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.337641:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b045c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.337646:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762833.337649:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b045c9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.337656:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b045c9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.337669:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b045c9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.337674:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b045c9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762833.337729:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015032448:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12396us (98103us total) trans 4294967949 rc 0/0 00000100:00100000:1.0:1789762833.337737:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 745 00000100:00100000:1.0:1789762833.337745:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356629079278 time 356629190795 state 0x2 ref 0 00000100:00100000:2.0:1789762833.339890:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762833.339925:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015032576 00000100:00100000:1.0:1789762833.339933:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015032576 00000100:00100000:1.0:1789762833.339966:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356629079278 time 356629190795 state 0x0 ref 1 00000100:00100000:1.0:1789762833.339980:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356729190795 time 356629190795 state 0x1 ref 1 00000100:00100000:2.0:1789762833.425397:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356729190795 time 356729291928 state 0x0 ref 1 00000100:00100000:2.0:1789762833.425423:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015032576:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762833.425482:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 20971520, mtime: 0x6aad9d11, atime: 0x0, ctime: 0x6aad9d11, blocks: 0xa007 00010000:00010000:2.0:1789762833.425489:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b045d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.425518:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=356s v=(0 0) 00010000:00010000:2.0:1789762833.425523:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762833.425530:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b045d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.425546:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b045d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762833.437589:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:3.0:1789762833.437594:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:3.0:1789762833.437603:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b045d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.437613:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b045d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.437620:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b045d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762833.437635:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967950 is committed 00010000:00010000:3.0:1789762833.437635:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762833.437642:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b045d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.437650:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b045d0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.437665:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b045d0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.437670:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b045d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762833.437730:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015032576:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12317us (97840us total) trans 4294967950 rc 0/0 00000100:00100000:3.0:1789762833.437742:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 746 00000100:00100000:3.0:1789762833.437754:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356729190795 time 356729291928 state 0x2 ref 0 00000100:00100000:0.0:1789762833.439925:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762833.439971:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015032704 00000100:00100000:2.0:1789762833.439979:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015032704 00000100:00100000:2.0:1789762833.440017:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356729190795 time 356729291928 state 0x0 ref 1 00000100:00100000:2.0:1789762833.440031:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356829291928 time 356729291928 state 0x1 ref 1 00000100:00100000:1.0:1789762833.506295:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762833.506331:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762833.506352:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015032832 00000100:00100000:1.0:1789762833.506353:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762833.506363:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015032832 00000100:00100000:0.0:1789762833.506382:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015032960 00000100:00100000:1.0:1789762833.506385:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762833.506402:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015032960 00000100:00100000:3.0:1789762833.506409:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015033216 00000100:00100000:1.0:1789762833.506415:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015033088 00000100:00100000:3.0:1789762833.506417:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015033216 00000100:00100000:2.0:1789762833.506422:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 312 00000100:00100000:1.0:1789762833.506423:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015033088 00000100:00100000:2.0:1789762833.506434:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85085c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015032832:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762833.506475:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 268 00000100:00100000:1.0:1789762833.506482:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 135 00000100:00100000:3.0:1789762833.506488:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 269 00000100:00100000:0.0:1789762833.506490:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702015032960:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762833.506502:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2365:x1876702015033088:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762833.506503:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85085c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015032832:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 66us (208us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762833.506508:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015033216:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762833.506524:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 312 00000100:00100000:0.0:1789762833.506539:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702015032960:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 55us (207us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762833.506550:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 268 00000100:00100000:1.0:1789762833.506551:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2365:x1876702015033088:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 54us (164us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762833.506560:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 135 00000100:00100000:3.0:1789762833.506566:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015033216:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 61us (208us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762833.506578:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762833.506584:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762833.506590:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 269 00000100:00100000:2.0:1789762833.525432:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356829291928 time 356829334020 state 0x0 ref 1 00000100:00100000:2.0:1789762833.525445:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015032704:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762833.525489:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 22020096, mtime: 0x6aad9d11, atime: 0x0, ctime: 0x6aad9d11, blocks: 0xa807 00010000:00010000:2.0:1789762833.525497:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecfe3bd3/0x416193e012b045d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.525513:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=356s v=(0 0) 00010000:00010000:2.0:1789762833.525518:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762833.525524:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ecfe3bd3/0x416193e012b045d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.525539:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ecfe3bd3/0x416193e012b045d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789762833.527210:0:6868:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 356831121059ns, next at 357831118457ns, rc = 0 00000100:00100000:2.0:1789762833.527228:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022172032:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762833.527267:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762833.527313:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022172032 00000100:00100000:1.0:1789762833.527319:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022172032 00000100:00080000:2.0:1789762833.527354:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762833.527365:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:1.0:1789762833.527373:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022172032:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789762833.527386:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528023552 pending 1703936 free 3839885312 avail 3818913792 00000100:00100000:1.0:1789762833.527433:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022172032:12345-0@lo:13:osp-pre-0-0.0 Request processed in 60us (163us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762833.527443:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00100000:2.0:1789762833.527454:0:3272:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967935 gen 2 00000004:00080000:2.0:1789762833.527459:0:3272:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebac6ed80, transno 4294967935 00000100:00100000:2.0:1789762833.527477:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022172032:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1789762833.536785:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967951 is committed 00002000:00010000:2.0:1789762833.536808:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 22020096 -> 23068672 00002000:00010000:2.0:1789762833.536813:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:2.0:1789762833.536823:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ecfe3bd3/0x416193e012b045d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.536832:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ecfe3bd3/0x416193e012b045d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.536849:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ecfe3bd3/0x416193e012b045d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.536855:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762833.536858:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecfe3bd3/0x416193e012b045d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.536864:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ecfe3bd3/0x416193e012b045d7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.536878:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ecfe3bd3/0x416193e012b045d7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.536884:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ecfe3bd3/0x416193e012b045d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762833.536932:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015032704:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11488us (97007us total) trans 4294967951 rc 0/0 00000100:00100000:2.0:1789762833.536939:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 747 00000100:00100000:2.0:1789762833.536948:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356829291928 time 356829334020 state 0x2 ref 0 00000100:00100000:2.0:1789762833.539163:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762833.539206:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015033344 00000100:00100000:0.0:1789762833.539213:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015033344 00000100:00100000:0.0:1789762833.539252:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356829291928 time 356829334020 state 0x0 ref 1 00000100:00100000:0.0:1789762833.539267:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356929334020 time 356829334020 state 0x1 ref 1 00000100:00100000:1.0:1789762833.625537:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356929334020 time 356929432518 state 0x0 ref 1 00000100:00100000:1.0:1789762833.625561:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015033344:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762833.625617:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 23068672, mtime: 0x6aad9d11, atime: 0x0, ctime: 0x6aad9d11, blocks: 0xb007 00010000:00010000:1.0:1789762833.625638:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b045de lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.625652:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=356s v=(0 0) 00010000:00010000:1.0:1789762833.625656:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762833.625660:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b045de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.625672:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b045de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762833.637053:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:3.0:1789762833.637060:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 45063 -> 47111 00000001:00080000:2.0:1789762833.637067:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967952 is committed 00010000:00010000:3.0:1789762833.637072:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b045de lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.637117:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b045de lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.637129:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b045de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.637136:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762833.637141:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b045de lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.637151:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b045de lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.637170:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b045de lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.637179:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b045de lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762833.637248:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015033344:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11694us (98084us total) trans 4294967952 rc 0/0 00000100:00100000:3.0:1789762833.637266:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 748 00000100:00100000:3.0:1789762833.637282:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356929334020 time 356929432518 state 0x2 ref 0 00000100:00100000:2.0:1789762833.639566:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762833.639629:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015033472 00000100:00100000:1.0:1789762833.639638:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015033472 00000100:00100000:1.0:1789762833.639677:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 356929334020 time 356929432518 state 0x0 ref 1 00000100:00100000:1.0:1789762833.639689:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357029432518 time 356929432518 state 0x1 ref 1 00000100:00100000:3.0:1789762833.725620:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357029432518 time 357029514648 state 0x0 ref 1 00000100:00100000:3.0:1789762833.725647:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015033472:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762833.725701:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 24117248, mtime: 0x6aad9d11, atime: 0x0, ctime: 0x6aad9d11, blocks: 0xb807 00010000:00010000:3.0:1789762833.725711:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b045e5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.725726:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=357s v=(0 0) 00010000:00010000:3.0:1789762833.725731:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762833.725738:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b045e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.725754:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b045e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762833.737426:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967953 is committed 00002000:00010000:2.0:1789762833.737466:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 24117248 -> 25165824 00002000:00010000:2.0:1789762833.737478:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:2.0:1789762833.737488:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b045e5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.737496:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b045e5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.737506:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b045e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.737514:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762833.737519:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b045e5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.737530:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b045e5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.737548:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b045e5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.737556:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b045e5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762833.737629:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015033472:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11989us (98061us total) trans 4294967953 rc 0/0 00000100:00100000:2.0:1789762833.737641:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 749 00000100:00100000:2.0:1789762833.737651:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357029432518 time 357029514648 state 0x2 ref 0 00000100:00100000:0.0:1789762833.739829:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762833.739907:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015033600 00000100:00100000:2.0:1789762833.739915:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015033600 00000100:00100000:2.0:1789762833.739953:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357029432518 time 357029514648 state 0x0 ref 1 00000100:00100000:2.0:1789762833.739964:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357129514648 time 357029514648 state 0x1 ref 1 00000100:00100000:2.0:1789762833.825710:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357129514648 time 357129605422 state 0x0 ref 1 00000100:00100000:2.0:1789762833.825735:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015033600:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762833.825784:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 25165824, mtime: 0x6aad9d11, atime: 0x0, ctime: 0x6aad9d11, blocks: 0xc007 00010000:00010000:2.0:1789762833.825792:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8e62cce/0x416193e012b045ec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.825805:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=357s v=(0 0) 00010000:00010000:2.0:1789762833.825809:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762833.825814:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8e62cce/0x416193e012b045ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762833.825825:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8e62cce/0x416193e012b045ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762833.837239:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967954 is committed 00002000:00010000:1.0:1789762833.837272:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:1.0:1789762833.837278:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:1.0:1789762833.837290:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b8e62cce/0x416193e012b045ec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.837306:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8e62cce/0x416193e012b045ec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.837313:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8e62cce/0x416193e012b045ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.837321:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1bc00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762833.837327:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8e62cce/0x416193e012b045ec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.837337:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8e62cce/0x416193e012b045ec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.837357:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8e62cce/0x416193e012b045ec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762833.837366:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8e62cce/0x416193e012b045ec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762833.837431:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015033600:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11704us (97602us total) trans 4294967954 rc 0/0 00000100:00100000:1.0:1789762833.837440:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 750 00000100:00100000:1.0:1789762833.837448:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357129514648 time 357129605422 state 0x2 ref 0 00000100:00100000:1.0:1789762833.839777:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762833.839824:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015033728 00000100:00100000:0.0:1789762833.839833:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015033728 00000100:00100000:0.0:1789762833.839879:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357129514648 time 357129605422 state 0x0 ref 1 00000100:00100000:0.0:1789762833.839896:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357229605422 time 357129605422 state 0x1 ref 1 00000100:00100000:3.0:1789762833.925809:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357229605422 time 357229701157 state 0x0 ref 1 00000100:00100000:3.0:1789762833.925836:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015033728:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762833.925890:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 26214400, mtime: 0x6aad9d11, atime: 0x0, ctime: 0x6aad9d11, blocks: 0xc807 00010000:00010000:3.0:1789762833.925899:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b045f3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.925927:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=357s v=(0 0) 00010000:00010000:3.0:1789762833.925931:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762833.925938:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b045f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762833.925952:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b045f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762833.937708:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 26214400 -> 27262976 00002000:00010000:0.0:1789762833.937714:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 1789762833 -> 1789762834 00002000:00010000:0.0:1789762833.937716:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 1789762833 -> 1789762834 00002000:00010000:0.0:1789762833.937718:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:0.0:1789762833.937729:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b045f3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762833.937746:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967955 is committed 00010000:00010000:0.0:1789762833.937760:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b045f3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.937768:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b045f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.937774:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762833.937777:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b045f3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.937783:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b045f3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.937798:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b045f3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762833.937803:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b045f3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762833.937876:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015033728:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12045us (98095us total) trans 4294967955 rc 0/0 00000100:00100000:0.0:1789762833.937888:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 751 00000100:00100000:0.0:1789762833.937902:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357229605422 time 357229701157 state 0x2 ref 0 00000100:00100000:1.0:1789762833.940407:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762833.940457:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015033856 00000100:00100000:3.0:1789762833.940467:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015033856 00000100:00100000:3.0:1789762833.940513:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357229605422 time 357229701157 state 0x0 ref 1 00000100:00100000:3.0:1789762833.940527:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357329701157 time 357229701157 state 0x1 ref 1 00000100:00100000:2.0:1789762834.025945:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357329701157 time 357329841033 state 0x0 ref 1 00000100:00100000:2.0:1789762834.026002:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015033856:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762834.026053:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 27262976, mtime: 0x6aad9d12, atime: 0x0, ctime: 0x6aad9d12, blocks: 0xd007 00010000:00010000:2.0:1789762834.026060:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000327ee1bc/0x416193e012b045fa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789762834.026069:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789762834.026074:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=357s v=(0 0) 00010000:00010000:2.0:1789762834.026079:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762834.026123:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000327ee1bc/0x416193e012b045fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.026138:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000327ee1bc/0x416193e012b045fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762834.038461:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967956 is committed 00002000:00010000:1.0:1789762834.038492:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:1.0:1789762834.038498:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:1.0:1789762834.038511:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000327ee1bc/0x416193e012b045fa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.038523:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000327ee1bc/0x416193e012b045fa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.038532:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000327ee1bc/0x416193e012b045fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.038540:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762834.038545:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000327ee1bc/0x416193e012b045fa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.038556:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000327ee1bc/0x416193e012b045fa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.038575:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000327ee1bc/0x416193e012b045fa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.038584:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000327ee1bc/0x416193e012b045fa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762834.038653:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015033856:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12670us (98245us total) trans 4294967956 rc 0/0 00000100:00100000:1.0:1789762834.038665:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 752 00000100:00100000:1.0:1789762834.038677:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357329701157 time 357329841033 state 0x2 ref 0 00000004:00080000:0.0:1789762834.039201:0:7726:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 357343113037ns, next at 358343111120ns, rc = 0 00000100:00100000:3.0:1789762834.039212:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022172288:0@lo:400:kworker.0 00000100:00100000:2.0:1789762834.039230:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e902dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022172160:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762834.039276:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762834.039281:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762834.039312:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022172288 00000100:00100000:1.0:1789762834.039315:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022172160 00000100:00100000:0.0:1789762834.039319:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022172288 00000100:00100000:1.0:1789762834.039322:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022172160 00000100:00100000:2.0:1789762834.039339:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022172672:0@lo:400:kworker.0 00000100:00100000:3.0:1789762834.039342:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022172800:0@lo:400:kworker.0 00000100:00100000:0.0:1789762834.039364:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 313 00000100:00100000:0.0:1789762834.039379:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022172288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762834.039379:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762834.039385:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 114 00000100:00100000:3.0:1789762834.039393:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762834.039403:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022172160:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762834.039416:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022172416:0@lo:400:kworker.0 00002000:00000024:1.0:1789762834.039432:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 97058816 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789762834.039439:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022172288:12345-0@lo:400:kworker.0 Request processed in 66us (164us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762834.039443:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022172288:0@lo:400:kworker.0 00000100:00100000:0.0:1789762834.039449:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 313 00000100:00100000:2.0:1789762834.039459:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789762834.039484:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762834.039498:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022172800 00000100:00100000:1.0:1789762834.039499:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022172160:12345-0@lo:13:osp-pre-1-0.0 Request processed in 100us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762834.039500:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022172928:0@lo:400:kworker.0 00000100:00100000:0.0:1789762834.039502:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022172800 00000100:00100000:1.0:1789762834.039515:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 114 00000100:00080000:3.0:1789762834.039539:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762834.039545:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022172416 00000100:00100000:2.0:1789762834.039546:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762834.039549:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022172416 00000100:00100000:0.0:1789762834.039576:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 270 00000100:00100000:2.0:1789762834.039579:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789762834.039582:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022172672 00000100:00100000:0.0:1789762834.039586:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022172800:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762834.039588:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022172544:0@lo:400:kworker.0 00000100:00100000:1.0:1789762834.039590:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022172672 00000100:00100000:3.0:1789762834.039599:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 271 00000100:00100000:3.0:1789762834.039614:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022172416:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762834.039629:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762834.039636:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022172928 00000100:00100000:0.0:1789762834.039642:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022172800:12345-0@lo:400:kworker.0 Request processed in 57us (250us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762834.039644:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022172928 00000100:00100000:0.0:1789762834.039651:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 270 00000100:00100000:0.0:1789762834.039669:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022172544 00000100:00100000:0.0:1789762834.039673:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022172544 00000100:00100000:3.0:1789762834.039678:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022172416:12345-0@lo:400:kworker.0 Request processed in 65us (217us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762834.039684:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 136 00000100:00100000:2.0:1789762834.039685:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022172416:0@lo:400:kworker.0 00000100:00100000:1.0:1789762834.039703:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022172672:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762834.039708:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 272 00000100:00100000:3.0:1789762834.039719:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 271 00000100:00100000:0.0:1789762834.039724:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad02680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022172544:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762834.039729:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 137 00000100:00100000:2.0:1789762834.039739:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022172928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762834.039760:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022172672:12345-0@lo:400:kworker.0 Request processed in 61us (382us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762834.039769:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 136 00000100:00100000:0.0:1789762834.039786:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad02680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022172544:12345-0@lo:400:kworker.0 Request processed in 67us (157us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762834.039790:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022172928:12345-0@lo:400:kworker.0 Request processed in 50us (246us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762834.039797:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 272 00000100:00100000:2.0:1789762834.039802:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000100:00100000:3.0:1789762834.039810:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022172544:0@lo:400:kworker.0 00000100:00080000:0.0:1789762834.039820:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762834.039831:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022172928:0@lo:400:kworker.0 00000100:00100000:0.0:1789762834.039850:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022172800:0@lo:400:kworker.0 00000100:00100000:2.0:1789762834.039857:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022172672:0@lo:400:kworker.0 00000100:00100000:2.0:1789762834.039883:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e902dd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022172160:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762834.040756:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762834.040799:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015033984 00000100:00100000:2.0:1789762834.040804:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015033984 00000100:00100000:2.0:1789762834.040832:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357329701157 time 357329841033 state 0x0 ref 1 00000100:00100000:2.0:1789762834.040845:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357429841033 time 357329841033 state 0x1 ref 1 00000100:00100000:1.0:1789762834.126043:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357429841033 time 357429933978 state 0x0 ref 1 00000100:00100000:1.0:1789762834.126070:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015033984:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762834.126158:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 28311552, mtime: 0x6aad9d12, atime: 0x0, ctime: 0x6aad9d12, blocks: 0xd807 00010000:00010000:1.0:1789762834.126167:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04601 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.126180:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=357s v=(0 0) 00010000:00010000:1.0:1789762834.126184:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762834.126189:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04601 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.126201:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04601 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762834.138066:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967957 is committed 00002000:00010000:1.0:1789762834.138121:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:1.0:1789762834.138128:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 55303 -> 57351 00010000:00010000:1.0:1789762834.138141:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04601 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.138154:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04601 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.138163:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04601 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.138169:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762834.138173:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04601 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.138179:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04601 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.138197:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04601 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.138203:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b04601 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762834.138259:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015033984:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12198us (97503us total) trans 4294967957 rc 0/0 00000100:00100000:1.0:1789762834.138267:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 753 00000100:00100000:1.0:1789762834.138276:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357429841033 time 357429933978 state 0x2 ref 0 00000100:00100000:3.0:1789762834.140472:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762834.140525:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015034112 00000100:00100000:0.0:1789762834.140534:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015034112 00000100:00100000:0.0:1789762834.140578:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357429841033 time 357429933978 state 0x0 ref 1 00000100:00100000:0.0:1789762834.140590:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357529933978 time 357429933978 state 0x1 ref 1 00000100:00100000:0.0:1789762834.226166:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357529933978 time 357530057810 state 0x0 ref 1 00000100:00100000:0.0:1789762834.226197:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4970700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015034112:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762834.226253:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 29360128, mtime: 0x6aad9d12, atime: 0x0, ctime: 0x6aad9d12, blocks: 0xe007 00010000:00010000:0.0:1789762834.226263:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04608 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.226279:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=357s v=(0 0) 00010000:00010000:0.0:1789762834.226283:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762834.226291:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04608 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762834.226304:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04608 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762834.239132:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967958 is committed 00002000:00010000:2.0:1789762834.239150:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:2.0:1789762834.239155:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 57351 -> 59399 00010000:00010000:2.0:1789762834.239167:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04608 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.239182:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04608 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.239193:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04608 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.239217:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbfc00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762834.239222:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04608 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.239234:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04608 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.239253:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04608 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.239262:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04608 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762834.239329:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4970700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015034112:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13141us (98854us total) trans 4294967958 rc 0/0 00000100:00100000:2.0:1789762834.239338:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 754 00000100:00100000:2.0:1789762834.239348:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357529933978 time 357530057810 state 0x2 ref 0 00000100:00100000:1.0:1789762834.241639:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762834.241685:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015034240 00000100:00100000:3.0:1789762834.241695:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015034240 00000100:00100000:3.0:1789762834.241739:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357529933978 time 357530057810 state 0x0 ref 1 00000100:00100000:3.0:1789762834.241754:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357630057810 time 357530057810 state 0x1 ref 1 00000100:00100000:1.0:1789762834.326261:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357630057810 time 357630156452 state 0x0 ref 1 00000100:00100000:1.0:1789762834.326287:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015034240:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762834.326346:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 30408704, mtime: 0x6aad9d12, atime: 0x0, ctime: 0x6aad9d12, blocks: 0xe807 00010000:00010000:1.0:1789762834.326357:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0460f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.326373:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=357s v=(0 0) 00010000:00010000:1.0:1789762834.326377:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762834.326382:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0460f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.326398:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0460f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762834.338616:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 30408704 -> 31457280 00002000:00010000:1.0:1789762834.338622:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 59399 -> 61447 00010000:00010000:1.0:1789762834.338634:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0460f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.338647:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0460f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.338656:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0460f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.338664:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762834.338669:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0460f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.338679:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0460f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.338699:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0460f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.338708:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0460f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762834.338784:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2364:x1876702015034240:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12503us (97142us total) trans 4294967959 rc 0/0 00000100:00100000:1.0:1789762834.338796:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 755 00000100:00100000:1.0:1789762834.338808:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357630057810 time 357630156452 state 0x2 ref 0 00000001:00080000:1.0:1789762834.338847:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967959 is committed 00000100:00100000:1.0:1789762834.341066:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762834.341129:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015034368 00000100:00100000:2.0:1789762834.341137:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015034368 00000100:00100000:2.0:1789762834.341184:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357630057810 time 357630156452 state 0x0 ref 1 00000100:00100000:2.0:1789762834.341199:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357730156452 time 357630156452 state 0x1 ref 1 00010000:00010000:2.0:1789762834.359226:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762834.426365:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357730156452 time 357730255925 state 0x0 ref 1 00000100:00100000:1.0:1789762834.426393:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015034368:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762834.426450:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 31457280, mtime: 0x6aad9d12, atime: 0x0, ctime: 0x6aad9d12, blocks: 0xf007 00010000:00010000:1.0:1789762834.426459:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04616 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.426476:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=357s v=(0 0) 00010000:00010000:1.0:1789762834.426481:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762834.426488:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04616 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.426502:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04616 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762834.438749:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967960 is committed 00002000:00010000:2.0:1789762834.438765:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:2.0:1789762834.438772:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 61447 -> 63495 00010000:00010000:2.0:1789762834.438786:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04616 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.438797:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04616 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.438804:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04616 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.438809:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762834.438812:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04616 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.438819:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04616 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.438834:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04616 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.438842:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04616 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762834.438926:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015034368:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12541us (97859us total) trans 4294967960 rc 0/0 00000100:00100000:2.0:1789762834.438937:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 756 00000100:00100000:2.0:1789762834.438955:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357730156452 time 357730255925 state 0x2 ref 0 00000100:00100000:3.0:1789762834.441448:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762834.441499:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015034496 00000100:00100000:0.0:1789762834.441507:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015034496 00000100:00100000:0.0:1789762834.441547:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357730156452 time 357730255925 state 0x0 ref 1 00000100:00100000:0.0:1789762834.441568:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357830255925 time 357730255925 state 0x1 ref 1 00000100:00100000:2.0:1789762834.526453:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357830255925 time 357830348442 state 0x0 ref 1 00000100:00100000:2.0:1789762834.526478:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015034496:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762834.526537:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 32505856, mtime: 0x6aad9d12, atime: 0x0, ctime: 0x6aad9d12, blocks: 0xf807 00010000:00010000:2.0:1789762834.526545:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0461d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.526558:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=357s v=(0 0) 00010000:00010000:2.0:1789762834.526562:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762834.526569:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0461d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.526584:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0461d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762834.538625:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967961 is committed 00002000:00010000:2.0:1789762834.538678:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:2.0:1789762834.538683:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 63495 -> 65543 00010000:00010000:2.0:1789762834.538693:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0461d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.538702:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0461d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.538709:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0461d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.538721:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ac00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762834.538725:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0461d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.538731:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0461d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.538745:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0461d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.538751:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0461d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762834.538813:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015034496:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12343us (97364us total) trans 4294967961 rc 0/0 00000100:00100000:2.0:1789762834.538820:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 757 00000100:00100000:2.0:1789762834.538829:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357830255925 time 357830348442 state 0x2 ref 0 00000100:00100000:3.0:1789762834.541137:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762834.541191:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015034624 00000100:00100000:1.0:1789762834.541200:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015034624 00000100:00100000:1.0:1789762834.541238:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357830255925 time 357830348442 state 0x0 ref 1 00000100:00100000:1.0:1789762834.541249:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357930348442 time 357830348442 state 0x1 ref 1 00000100:00100000:1.0:1789762834.626517:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357930348442 time 357930411858 state 0x0 ref 1 00000100:00100000:1.0:1789762834.626539:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015034624:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762834.626591:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 33554432, mtime: 0x6aad9d12, atime: 0x0, ctime: 0x6aad9d12, blocks: 0x10007 00010000:00010000:1.0:1789762834.626598:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b04624 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.626612:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=357s v=(0 0) 00010000:00010000:1.0:1789762834.626616:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762834.626621:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b04624 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.626632:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b04624 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762834.638118:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967962 is committed 00002000:00010000:2.0:1789762834.638133:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:2.0:1789762834.638140:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 65543 -> 67591 00010000:00010000:2.0:1789762834.638151:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b04624 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.638160:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b04624 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.638177:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b04624 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.638183:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762834.638186:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b04624 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.638193:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b04624 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.638207:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b04624 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.638213:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b04624 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762834.638273:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015034624:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11738us (97136us total) trans 4294967962 rc 0/0 00000100:00100000:2.0:1789762834.638283:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 758 00000100:00100000:2.0:1789762834.638295:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357930348442 time 357930411858 state 0x2 ref 0 00000100:00100000:3.0:1789762834.640619:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762834.640690:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015034752 00000100:00100000:2.0:1789762834.640698:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015034752 00000100:00100000:2.0:1789762834.640738:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 357930348442 time 357930411858 state 0x0 ref 1 00000100:00100000:2.0:1789762834.640759:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358030411858 time 357930411858 state 0x1 ref 1 00000100:00100000:2.0:1789762834.726585:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358030411858 time 358030480438 state 0x0 ref 1 00000100:00100000:2.0:1789762834.726609:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015034752:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762834.726657:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 34603008, mtime: 0x6aad9d12, atime: 0x0, ctime: 0x6aad9d12, blocks: 0x10807 00010000:00010000:2.0:1789762834.726665:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0462b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.726678:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=358s v=(0 0) 00010000:00010000:2.0:1789762834.726682:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762834.726687:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0462b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.726700:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0462b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762834.741807:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967963 is committed 00002000:00010000:2.0:1789762834.741810:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:2.0:1789762834.741820:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 67591 -> 69639 00010000:00010000:2.0:1789762834.741832:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0462b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.741846:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0462b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.741872:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0462b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.741881:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ac00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762834.741886:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0462b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.741893:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0462b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.741908:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0462b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.741913:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b0462b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762834.741973:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015034752:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15370us (101354us total) trans 4294967963 rc 0/0 00000100:00100000:2.0:1789762834.741980:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 759 00000100:00100000:2.0:1789762834.741990:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358030411858 time 358030480438 state 0x2 ref 0 00000100:00100000:0.0:1789762834.744451:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762834.744490:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015034880 00000100:00100000:2.0:1789762834.744499:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015034880 00000100:00100000:2.0:1789762834.744546:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358030411858 time 358030480438 state 0x0 ref 1 00000100:00100000:2.0:1789762834.744566:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358130480438 time 358030480438 state 0x1 ref 1 00000100:00100000:2.0:1789762834.826649:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358130480438 time 358130538627 state 0x0 ref 1 00000100:00100000:2.0:1789762834.826676:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015034880:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762834.826749:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 35651584, mtime: 0x6aad9d12, atime: 0x0, ctime: 0x6aad9d12, blocks: 0x11007 00010000:00010000:2.0:1789762834.826758:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04632 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.826776:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=358s v=(0 0) 00010000:00010000:2.0:1789762834.826780:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762834.826786:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04632 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.826800:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04632 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762834.839943:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967964 is committed 00002000:00010000:2.0:1789762834.839971:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 35651584 -> 36700160 00002000:00010000:2.0:1789762834.839978:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 69639 -> 71687 00010000:00010000:2.0:1789762834.839990:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04632 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.840000:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04632 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.840020:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04632 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.840026:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ac00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762834.840030:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04632 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.840037:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04632 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.840054:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04632 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.840061:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04632 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762834.840193:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015034880:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13522us (95739us total) trans 4294967964 rc 0/0 00000100:00100000:2.0:1789762834.840207:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 760 00000100:00100000:2.0:1789762834.840218:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358130480438 time 358130538627 state 0x2 ref 0 00000100:00100000:0.0:1789762834.842656:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762834.842715:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015035008 00000100:00100000:1.0:1789762834.842727:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015035008 00000100:00100000:1.0:1789762834.842784:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358130480438 time 358130538627 state 0x0 ref 1 00000100:00100000:1.0:1789762834.842800:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358230538627 time 358130538627 state 0x1 ref 1 00000100:00100000:2.0:1789762834.926763:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358230538627 time 358230651507 state 0x0 ref 1 00000100:00100000:2.0:1789762834.926803:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015035008:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762834.926908:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 36700160, mtime: 0x6aad9d12, atime: 0x0, ctime: 0x6aad9d12, blocks: 0x11807 00010000:00010000:2.0:1789762834.926918:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04639 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.926936:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=358s v=(0 0) 00010000:00010000:2.0:1789762834.926940:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762834.926946:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04639 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762834.926960:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04639 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762834.940040:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967965 is committed 00002000:00010000:1.0:1789762834.940066:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 36700160 -> 37748736 00002000:00010000:1.0:1789762834.940073:0:14912:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 1789762834 -> 1789762835 00002000:00010000:1.0:1789762834.940076:0:14912:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 1789762834 -> 1789762835 00002000:00010000:1.0:1789762834.940079:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 71687 -> 73735 00010000:00010000:1.0:1789762834.940126:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04639 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.940143:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04639 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.940154:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04639 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.940163:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ac00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762834.940168:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04639 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.940180:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04639 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.940199:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04639 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762834.940208:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7a21dd8/0x416193e012b04639 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762834.940290:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015035008:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13503us (97633us total) trans 4294967965 rc 0/0 00000100:00100000:1.0:1789762834.940301:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 761 00000100:00100000:1.0:1789762834.940316:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358230538627 time 358230651507 state 0x2 ref 0 00000100:00100000:3.0:1789762834.942340:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762834.942391:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015035136 00000100:00100000:2.0:1789762834.942400:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015035136 00000100:00100000:2.0:1789762834.942447:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358230538627 time 358230651507 state 0x0 ref 1 00000100:00100000:2.0:1789762834.942459:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358330651507 time 358230651507 state 0x1 ref 1 00000100:00100000:2.0:1789762835.026829:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358330651507 time 358330724250 state 0x0 ref 1 00000100:00100000:2.0:1789762835.026856:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015035136:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762835.026914:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 37748736, mtime: 0x6aad9d13, atime: 0x0, ctime: 0x6aad9d13, blocks: 0x12007 00010000:00010000:2.0:1789762835.026922:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04640 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.026951:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=358s v=(0 0) 00010000:00010000:2.0:1789762835.026955:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762835.026960:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04640 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.026972:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04640 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762835.039699:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967966 is committed 00002000:00010000:1.0:1789762835.039716:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:1.0:1789762835.039724:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 73735 -> 75783 00010000:00010000:1.0:1789762835.039736:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04640 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.039751:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04640 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.039761:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04640 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.039769:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ae00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762835.039775:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04640 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.039785:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04640 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.039802:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04640 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.039817:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04640 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762835.039880:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015035136:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13032us (97538us total) trans 4294967966 rc 0/0 00000100:00100000:1.0:1789762835.039891:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 762 00000100:00100000:1.0:1789762835.039902:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358330651507 time 358330724250 state 0x2 ref 0 00000100:00100000:3.0:1789762835.041987:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762835.042050:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015035264 00000100:00100000:2.0:1789762835.042058:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015035264 00000100:00100000:2.0:1789762835.042125:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358330651507 time 358330724250 state 0x0 ref 1 00000100:00100000:2.0:1789762835.042139:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358430724250 time 358330724250 state 0x1 ref 1 00000001:00080000:0.0:1789762835.081791:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967415 is committed 00000001:00080000:0.0:1789762835.081804:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967412 is committed 00000001:00080000:0.0:1789762835.081832:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967414 is committed 00000100:00100000:2.0:1789762835.126899:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358430724250 time 358430792253 state 0x0 ref 1 00000100:00100000:2.0:1789762835.126927:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4970380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015035264:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762835.126984:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 38797312, mtime: 0x6aad9d13, atime: 0x0, ctime: 0x6aad9d13, blocks: 0x12807 00010000:00010000:2.0:1789762835.126994:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b04647 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.127019:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=358s v=(0 0) 00010000:00010000:2.0:1789762835.127024:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762835.127031:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b04647 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.127058:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b04647 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762835.139928:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:0.0:1789762835.139935:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 75783 -> 77831 00000001:00080000:3.0:1789762835.139943:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967967 is committed 00010000:00010000:0.0:1789762835.139948:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b04647 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.139963:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b04647 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.139971:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b04647 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.139976:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762835.139980:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b04647 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.139986:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b04647 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.140001:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b04647 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.140013:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b04647 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762835.140080:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4970380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015035264:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13159us (98093us total) trans 4294967967 rc 0/0 00000100:00100000:0.0:1789762835.140137:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 763 00000100:00100000:0.0:1789762835.140149:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358430724250 time 358430792253 state 0x2 ref 0 00000100:00100000:3.0:1789762835.142524:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762835.142576:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015035392 00000100:00100000:1.0:1789762835.142592:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015035392 00000100:00100000:1.0:1789762835.142631:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358430724250 time 358430792253 state 0x0 ref 1 00000100:00100000:1.0:1789762835.142642:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358530792253 time 358430792253 state 0x1 ref 1 00000100:00100000:1.0:1789762835.226989:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358530792253 time 358530883580 state 0x0 ref 1 00000100:00100000:1.0:1789762835.227016:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4970e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015035392:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762835.227075:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 39845888, mtime: 0x6aad9d13, atime: 0x0, ctime: 0x6aad9d13, blocks: 0x13007 00010000:00010000:1.0:1789762835.227123:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0464e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.227139:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=358s v=(0 0) 00010000:00010000:1.0:1789762835.227142:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762835.227148:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0464e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.227159:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0464e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762835.241591:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967968 is committed 00002000:00010000:1.0:1789762835.241606:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 39845888 -> 40894464 00002000:00010000:1.0:1789762835.241615:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 77831 -> 79879 00010000:00010000:1.0:1789762835.241629:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0464e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.241644:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0464e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.241655:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0464e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.241664:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762835.241669:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0464e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.241676:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0464e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.241692:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0464e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.241701:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0464e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762835.241778:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4970e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015035392:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14771us (99254us total) trans 4294967968 rc 0/0 00000100:00100000:1.0:1789762835.241789:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 764 00000100:00100000:1.0:1789762835.241799:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358530792253 time 358530883580 state 0x2 ref 0 00000100:00100000:3.0:1789762835.244173:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762835.244241:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015035520 00000100:00100000:2.0:1789762835.244251:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015035520 00000100:00100000:2.0:1789762835.244303:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358530792253 time 358530883580 state 0x0 ref 1 00000100:00100000:2.0:1789762835.244319:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358630883580 time 358530883580 state 0x1 ref 1 00000100:00100000:1.0:1789762835.327150:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358630883580 time 358631045512 state 0x0 ref 1 00000100:00100000:1.0:1789762835.327176:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015035520:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762835.327226:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 40894464, mtime: 0x6aad9d13, atime: 0x0, ctime: 0x6aad9d13, blocks: 0x13807 00010000:00010000:1.0:1789762835.327234:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04655 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.327247:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=358s v=(0 0) 00010000:00010000:1.0:1789762835.327251:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762835.327255:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04655 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.327266:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04655 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762835.342276:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967969 is committed 00002000:00010000:2.0:1789762835.342288:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:2.0:1789762835.342296:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 79879 -> 81927 00010000:00010000:2.0:1789762835.342310:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04655 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.342324:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04655 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.342335:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04655 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.342342:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762835.342348:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04655 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.342358:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04655 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.342389:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04655 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.342398:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04655 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762835.342479:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015035520:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15310us (98306us total) trans 4294967969 rc 0/0 00000100:00100000:2.0:1789762835.342491:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 765 00000100:00100000:2.0:1789762835.342503:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358630883580 time 358631045512 state 0x2 ref 0 00000100:00100000:3.0:1789762835.344900:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762835.344954:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015035648 00000100:00100000:2.0:1789762835.344965:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015035648 00000100:00100000:2.0:1789762835.345012:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358630883580 time 358631045512 state 0x0 ref 1 00000100:00100000:2.0:1789762835.345028:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358731045512 time 358631045512 state 0x1 ref 1 00010000:00010000:3.0:1789762835.383247:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762835.427255:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358731045512 time 358731146104 state 0x0 ref 1 00000100:00100000:0.0:1789762835.427284:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4972a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015035648:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762835.427357:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 41943040, mtime: 0x6aad9d13, atime: 0x0, ctime: 0x6aad9d13, blocks: 0x14007 00010000:00010000:0.0:1789762835.427370:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b0465c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.427392:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=358s v=(0 0) 00010000:00010000:0.0:1789762835.427398:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762835.427406:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b0465c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762835.427425:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b0465c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762835.440690:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967970 is committed 00002000:00010000:2.0:1789762835.440718:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 41943040 -> 42991616 00002000:00010000:2.0:1789762835.440725:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 81927 -> 83975 00010000:00010000:2.0:1789762835.440737:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b0465c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.440748:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b0465c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.440769:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b0465c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.440778:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762835.440783:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b0465c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.440793:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b0465c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.440816:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b0465c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.440825:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b0465c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762835.440912:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4972a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015035648:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13635us (96010us total) trans 4294967970 rc 0/0 00000100:00100000:2.0:1789762835.440924:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 766 00000100:00100000:2.0:1789762835.440939:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358731045512 time 358731146104 state 0x2 ref 0 00000100:00100000:2.0:1789762835.443648:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762835.443697:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015035776 00000100:00100000:1.0:1789762835.443706:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015035776 00000100:00100000:1.0:1789762835.443749:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358731045512 time 358731146104 state 0x0 ref 1 00000100:00100000:1.0:1789762835.443761:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358831146104 time 358731146104 state 0x1 ref 1 00000100:00100000:1.0:1789762835.527318:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358831146104 time 358831205569 state 0x0 ref 1 00000100:00100000:1.0:1789762835.527342:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015035776:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762835.527398:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 42991616, mtime: 0x6aad9d13, atime: 0x0, ctime: 0x6aad9d13, blocks: 0x14807 00010000:00010000:1.0:1789762835.527408:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b04663 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.527426:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=358s v=(0 0) 00010000:00010000:1.0:1789762835.527430:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762835.527436:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b04663 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762835.527447:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b04663 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762835.539664:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967971 is committed 00002000:00010000:2.0:1789762835.539702:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 42991616 -> 44040192 00002000:00010000:2.0:1789762835.539707:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 83975 -> 86023 00010000:00010000:2.0:1789762835.539716:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b04663 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.539731:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b04663 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.539739:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b04663 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.539748:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762835.539753:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b04663 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.539762:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b04663 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.539782:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b04663 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.539792:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b04663 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762835.539854:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015035776:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12520us (96206us total) trans 4294967971 rc 0/0 00000100:00100000:2.0:1789762835.539864:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 767 00000100:00100000:2.0:1789762835.539875:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358831146104 time 358831205569 state 0x2 ref 0 00000100:00100000:3.0:1789762835.542145:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762835.542179:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015035904 00000100:00100000:2.0:1789762835.542193:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015035904 00000100:00100000:2.0:1789762835.542227:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358831146104 time 358831205569 state 0x0 ref 1 00000100:00100000:2.0:1789762835.542238:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358931205569 time 358831205569 state 0x1 ref 1 00000100:00100000:2.0:1789762835.627371:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358931205569 time 358931268175 state 0x0 ref 1 00000100:00100000:2.0:1789762835.627397:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4972300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015035904:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762835.627456:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 44040192, mtime: 0x6aad9d13, atime: 0x0, ctime: 0x6aad9d13, blocks: 0x15007 00010000:00010000:2.0:1789762835.627463:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0466a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.627476:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=358s v=(0 0) 00010000:00010000:2.0:1789762835.627480:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762835.627489:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0466a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.627500:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0466a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762835.640457:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967972 is committed 00002000:00010000:2.0:1789762835.640532:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 44040192 -> 45088768 00002000:00010000:2.0:1789762835.640538:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 86023 -> 88071 00010000:00010000:2.0:1789762835.640548:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0466a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.640557:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0466a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.640572:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0466a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.640577:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762835.640580:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0466a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.640586:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0466a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.640600:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0466a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.640605:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0466a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762835.640665:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4972300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015035904:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13276us (98518us total) trans 4294967972 rc 0/0 00000100:00100000:2.0:1789762835.640675:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 768 00000100:00100000:2.0:1789762835.640684:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358931205569 time 358931268175 state 0x2 ref 0 00000100:00100000:3.0:1789762835.643005:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762835.643049:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015036032 00000100:00100000:2.0:1789762835.643057:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015036032 00000100:00100000:2.0:1789762835.643124:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 358931205569 time 358931268175 state 0x0 ref 1 00000100:00100000:2.0:1789762835.643144:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359031268175 time 358931268175 state 0x1 ref 1 00000100:00100000:2.0:1789762835.727421:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359031268175 time 359031316477 state 0x0 ref 1 00000100:00100000:2.0:1789762835.727444:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015036032:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762835.727491:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 45088768, mtime: 0x6aad9d13, atime: 0x0, ctime: 0x6aad9d13, blocks: 0x15807 00010000:00010000:2.0:1789762835.727499:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04671 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.727512:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=359s v=(0 0) 00010000:00010000:2.0:1789762835.727516:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762835.727520:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04671 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.727533:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04671 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762835.739003:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967973 is committed 00002000:00010000:2.0:1789762835.739012:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 45088768 -> 46137344 00002000:00010000:2.0:1789762835.739017:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 88071 -> 90121 00010000:00010000:2.0:1789762835.739027:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04671 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.739036:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04671 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.739054:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04671 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.739059:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762835.739063:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04671 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.739070:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04671 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.739084:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04671 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.739120:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04671 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762835.739175:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015036032:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11739us (96170us total) trans 4294967973 rc 0/0 00000100:00100000:2.0:1789762835.739184:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 769 00000100:00100000:2.0:1789762835.739193:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359031268175 time 359031316477 state 0x2 ref 0 00000100:00100000:3.0:1789762835.741254:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762835.741305:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015036160 00000100:00100000:1.0:1789762835.741312:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015036160 00000100:00100000:1.0:1789762835.741349:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359031268175 time 359031316477 state 0x0 ref 1 00000100:00100000:1.0:1789762835.741362:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359131316477 time 359031316477 state 0x1 ref 1 00000100:00100000:2.0:1789762835.827512:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359131316477 time 359131408324 state 0x0 ref 1 00000100:00100000:2.0:1789762835.827536:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015036160:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762835.827597:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 46137344, mtime: 0x6aad9d13, atime: 0x0, ctime: 0x6aad9d13, blocks: 0x16009 00010000:00010000:2.0:1789762835.827606:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04678 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.827622:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=359s v=(0 0) 00010000:00010000:2.0:1789762835.827628:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762835.827635:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04678 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.827652:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04678 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762835.839751:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967974 is committed 00002000:00010000:2.0:1789762835.839778:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 46137344 -> 47185920 00002000:00010000:2.0:1789762835.839785:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 90121 -> 92169 00010000:00010000:2.0:1789762835.839798:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04678 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.839808:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04678 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.839814:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04678 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.839819:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762835.839823:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04678 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.839841:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04678 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.839856:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04678 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.839861:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b04678 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762835.839931:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839eb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015036160:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12401us (98675us total) trans 4294967974 rc 0/0 00000100:00100000:2.0:1789762835.839939:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 770 00000100:00100000:2.0:1789762835.839948:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359131316477 time 359131408324 state 0x2 ref 0 00000100:00100000:3.0:1789762835.842163:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762835.842204:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015036288 00000100:00100000:2.0:1789762835.842214:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015036288 00000100:00100000:2.0:1789762835.842255:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359131316477 time 359131408324 state 0x0 ref 1 00000100:00100000:2.0:1789762835.842281:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359231408324 time 359131408324 state 0x1 ref 1 00000100:00100000:2.0:1789762835.927578:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359231408324 time 359231473940 state 0x0 ref 1 00000100:00100000:2.0:1789762835.927599:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015036288:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762835.927659:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 47185920, mtime: 0x6aad9d13, atime: 0x0, ctime: 0x6aad9d13, blocks: 0x16809 00010000:00010000:2.0:1789762835.927666:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0467f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.927679:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=359s v=(0 0) 00010000:00010000:2.0:1789762835.927683:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762835.927689:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0467f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762835.927699:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0467f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762835.941217:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967975 is committed 00002000:00010000:3.0:1789762835.941221:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 47185920 -> 48234496 00002000:00010000:3.0:1789762835.941226:0:6654:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 1789762835 -> 1789762836 00002000:00010000:3.0:1789762835.941231:0:6654:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 1789762835 -> 1789762836 00002000:00010000:3.0:1789762835.941232:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 92169 -> 94217 00010000:00010000:3.0:1789762835.941244:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0467f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.941255:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0467f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.941262:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0467f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.941268:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762835.941272:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0467f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.941279:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0467f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.941306:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0467f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762835.941311:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0467f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762835.941387:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015036288:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13791us (99223us total) trans 4294967975 rc 0/0 00000100:00100000:3.0:1789762835.941395:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 771 00000100:00100000:3.0:1789762835.941406:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359231408324 time 359231473940 state 0x2 ref 0 00000100:00100000:3.0:1789762835.943388:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762835.943429:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015036416 00000100:00100000:2.0:1789762835.943438:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015036416 00000100:00100000:2.0:1789762835.943481:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359231408324 time 359231473940 state 0x0 ref 1 00000100:00100000:2.0:1789762835.943497:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359331473940 time 359231473940 state 0x1 ref 1 00000100:00100000:2.0:1789762836.027648:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359331473940 time 359331538599 state 0x0 ref 1 00000100:00100000:2.0:1789762836.027700:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015036416:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:0.0:1789762836.027725:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762836.027761:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 48234496, mtime: 0x6aad9d14, atime: 0x0, ctime: 0x6aad9d14, blocks: 0x17009 00010000:00010000:2.0:1789762836.027773:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04686 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762836.027812:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=359s v=(0 0) 00010000:00010000:2.0:1789762836.027817:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762836.027825:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04686 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762836.027840:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04686 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762836.039068:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967976 is committed 00002000:00010000:1.0:1789762836.039133:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 48234496 -> 49283072 00002000:00010000:1.0:1789762836.039140:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 94217 -> 96265 00010000:00010000:1.0:1789762836.039154:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04686 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.039167:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04686 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.039178:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04686 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.039187:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ba00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762836.039193:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04686 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.039204:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04686 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.039221:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04686 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.039226:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04686 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.039303:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015036416:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11632us (95913us total) trans 4294967976 rc 0/0 00000100:00100000:1.0:1789762836.039313:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 772 00000100:00100000:1.0:1789762836.039322:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359331473940 time 359331538599 state 0x2 ref 0 00000100:00100000:3.0:1789762836.041552:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762836.041591:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015036544 00000100:00100000:1.0:1789762836.041600:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015036544 00000100:00100000:1.0:1789762836.041638:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359331473940 time 359331538599 state 0x0 ref 1 00000100:00100000:1.0:1789762836.041650:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359431538599 time 359331538599 state 0x1 ref 1 00000100:00100000:2.0:1789762836.127740:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359431538599 time 359431635590 state 0x0 ref 1 00000100:00100000:2.0:1789762836.127763:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015036544:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762836.127814:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 49283072, mtime: 0x6aad9d14, atime: 0x0, ctime: 0x6aad9d14, blocks: 0x17809 00010000:00010000:2.0:1789762836.127821:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b0468d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762836.127835:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=359s v=(0 0) 00010000:00010000:2.0:1789762836.127838:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762836.127843:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b0468d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762836.127856:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b0468d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762836.139611:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967977 is committed 00002000:00010000:1.0:1789762836.139666:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 49283072 -> 50331648 00002000:00010000:1.0:1789762836.139671:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 96265 -> 98313 00010000:00010000:1.0:1789762836.139681:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b0468d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.139690:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b0468d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.139697:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b0468d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.139702:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762836.139705:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b0468d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.139713:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b0468d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.139726:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b0468d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.139731:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a420710d/0x416193e012b0468d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.139787:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839ead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015036544:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12030us (98235us total) trans 4294967977 rc 0/0 00000100:00100000:1.0:1789762836.139795:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 773 00000100:00100000:1.0:1789762836.139803:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359431538599 time 359431635590 state 0x2 ref 0 00000100:00100000:3.0:1789762836.142063:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762836.142132:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015036672 00000100:00100000:2.0:1789762836.142147:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015036672 00000100:00100000:2.0:1789762836.142183:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359431538599 time 359431635590 state 0x0 ref 1 00000100:00100000:2.0:1789762836.142194:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359531635590 time 359431635590 state 0x1 ref 1 00000100:00100000:3.0:1789762836.227840:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359531635590 time 359531735280 state 0x0 ref 1 00000100:00100000:3.0:1789762836.227865:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015036672:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762836.227914:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 50331648, mtime: 0x6aad9d14, atime: 0x0, ctime: 0x6aad9d14, blocks: 0x18009 00010000:00010000:3.0:1789762836.227921:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04694 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.227934:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=359s v=(0 0) 00010000:00010000:3.0:1789762836.227937:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762836.227942:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04694 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762836.227966:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04694 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762836.239449:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 50331648 -> 51380224 00002000:00010000:0.0:1789762836.239455:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 98313 -> 100361 00010000:00010000:0.0:1789762836.239468:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04694 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.239482:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04694 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762836.239488:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967978 is committed 00010000:00010000:0.0:1789762836.239493:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04694 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.239503:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762836.239509:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04694 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.239521:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04694 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.239539:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04694 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.239545:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04694 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762836.239603:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839eb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015036672:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11746us (97539us total) trans 4294967978 rc 0/0 00000100:00100000:0.0:1789762836.239621:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 774 00000100:00100000:0.0:1789762836.239632:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359531635590 time 359531735280 state 0x2 ref 0 00000100:00100000:3.0:1789762836.241649:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762836.241693:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015036800 00000100:00100000:2.0:1789762836.241700:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015036800 00000100:00100000:2.0:1789762836.241735:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359531635590 time 359531735280 state 0x0 ref 1 00000100:00100000:2.0:1789762836.241746:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359631735280 time 359531735280 state 0x1 ref 1 00000100:00100000:1.0:1789762836.327927:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359631735280 time 359631822320 state 0x0 ref 1 00000100:00100000:1.0:1789762836.327950:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015036800:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762836.327999:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 51380224, mtime: 0x6aad9d14, atime: 0x0, ctime: 0x6aad9d14, blocks: 0x18809 00010000:00010000:1.0:1789762836.328006:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0469b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.328020:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=359s v=(0 0) 00010000:00010000:1.0:1789762836.328023:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.328029:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0469b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.328040:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0469b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762836.339595:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967979 is committed 00002000:00010000:1.0:1789762836.339653:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 51380224 -> 52428800 00002000:00010000:1.0:1789762836.339658:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 100361 -> 102409 00010000:00010000:1.0:1789762836.339667:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0469b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.339675:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0469b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.339682:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0469b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.339687:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762836.339690:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0469b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.339696:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0469b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.339710:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0469b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.339719:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b0469b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.339780:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839eaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015036800:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11837us (98130us total) trans 4294967979 rc 0/0 00000100:00100000:1.0:1789762836.339787:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 775 00000100:00100000:1.0:1789762836.339795:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359631735280 time 359631822320 state 0x2 ref 0 00000100:00100000:3.0:1789762836.341790:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762836.341864:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015036928 00000100:00100000:1.0:1789762836.341874:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015036928 00000100:00100000:1.0:1789762836.341916:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359631735280 time 359631822320 state 0x0 ref 1 00000100:00100000:1.0:1789762836.341928:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359731822320 time 359631822320 state 0x1 ref 1 00010000:00010000:0.0:1789762836.407277:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762836.427980:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359731822320 time 359731878600 state 0x0 ref 1 00000100:00100000:1.0:1789762836.428007:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015036928:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762836.428066:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 52428800, mtime: 0x6aad9d14, atime: 0x0, ctime: 0x6aad9d14, blocks: 0x19009 00010000:00010000:1.0:1789762836.428074:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046a2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.428125:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=359s v=(0 0) 00010000:00010000:1.0:1789762836.428132:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.428140:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.428154:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762836.439480:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967980 is committed 00002000:00010000:1.0:1789762836.439491:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 52428800 -> 53477376 00002000:00010000:1.0:1789762836.439496:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 102409 -> 104457 00010000:00010000:1.0:1789762836.439505:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046a2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.439514:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046a2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.439520:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.439525:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762836.439529:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046a2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.439536:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046a2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.439550:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046a2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.439560:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046a2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.439617:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015036928:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11621us (97826us total) trans 4294967980 rc 0/0 00000100:00100000:1.0:1789762836.439625:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 776 00000100:00100000:1.0:1789762836.439634:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359731822320 time 359731878600 state 0x2 ref 0 00000100:00100000:3.0:1789762836.441840:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762836.441890:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015037056 00000100:00100000:2.0:1789762836.441900:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015037056 00000100:00100000:2.0:1789762836.441943:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359731822320 time 359731878600 state 0x0 ref 1 00000100:00100000:2.0:1789762836.441955:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359831878600 time 359731878600 state 0x1 ref 1 00000100:00100000:0.0:1789762836.528161:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359831878600 time 359832052644 state 0x0 ref 1 00000100:00100000:0.0:1789762836.528194:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015037056:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762836.528253:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 53477376, mtime: 0x6aad9d14, atime: 0x0, ctime: 0x6aad9d14, blocks: 0x19809 00010000:00010000:0.0:1789762836.528264:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096b81728/0x416193e012b046a9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.528283:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=359s v=(0 0) 00010000:00010000:0.0:1789762836.528289:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762836.528296:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096b81728/0x416193e012b046a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762836.528312:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096b81728/0x416193e012b046a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762836.541250:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967981 is committed 00002000:00010000:1.0:1789762836.541262:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:1.0:1789762836.541268:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 104457 -> 106505 00010000:00010000:1.0:1789762836.541278:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000096b81728/0x416193e012b046a9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.541288:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096b81728/0x416193e012b046a9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.541294:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096b81728/0x416193e012b046a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.541299:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbfa00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762836.541303:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096b81728/0x416193e012b046a9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.541309:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096b81728/0x416193e012b046a9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.541327:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096b81728/0x416193e012b046a9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.541333:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096b81728/0x416193e012b046a9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.541395:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839ea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015037056:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13213us (99553us total) trans 4294967981 rc 0/0 00000100:00100000:1.0:1789762836.541403:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 777 00000100:00100000:1.0:1789762836.541412:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359831878600 time 359832052644 state 0x2 ref 0 00000100:00100000:3.0:1789762836.543551:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762836.543604:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015037184 00000100:00100000:2.0:1789762836.543613:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015037184 00000100:00100000:2.0:1789762836.543650:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359831878600 time 359832052644 state 0x0 ref 1 00000100:00100000:2.0:1789762836.543661:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359932052644 time 359832052644 state 0x1 ref 1 00000100:00100000:1.0:1789762836.628253:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359932052644 time 359932148822 state 0x0 ref 1 00000100:00100000:1.0:1789762836.628279:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015037184:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762836.628331:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 54525952, mtime: 0x6aad9d14, atime: 0x0, ctime: 0x6aad9d14, blocks: 0x1a009 00010000:00010000:1.0:1789762836.628338:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.628351:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=359s v=(0 0) 00010000:00010000:1.0:1789762836.628355:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.628359:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.628371:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762836.640435:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967982 is committed 00002000:00010000:1.0:1789762836.640453:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 54525952 -> 55574528 00002000:00010000:1.0:1789762836.640458:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 106505 -> 108553 00010000:00010000:1.0:1789762836.640471:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.640485:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.640495:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.640504:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762836.640510:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.640519:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.640535:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.640541:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.640603:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839eb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015037184:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12332us (97050us total) trans 4294967982 rc 0/0 00000100:00100000:1.0:1789762836.640615:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 778 00000100:00100000:1.0:1789762836.640629:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359932052644 time 359932148822 state 0x2 ref 0 00000100:00100000:0.0:1789762836.642749:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762836.642789:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015037312 00000100:00100000:1.0:1789762836.642799:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015037312 00000100:00100000:1.0:1789762836.642835:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 359932052644 time 359932148822 state 0x0 ref 1 00000100:00100000:1.0:1789762836.642847:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360032148822 time 359932148822 state 0x1 ref 1 00000100:00100000:1.0:1789762836.728339:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360032148822 time 360032233752 state 0x0 ref 1 00000100:00100000:1.0:1789762836.728365:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015037312:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762836.728425:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 55574528, mtime: 0x6aad9d14, atime: 0x0, ctime: 0x6aad9d14, blocks: 0x1a809 00010000:00010000:1.0:1789762836.728432:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.728445:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=360s v=(0 0) 00010000:00010000:1.0:1789762836.728449:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.728453:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.728465:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762836.742866:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967983 is committed 00002000:00010000:1.0:1789762836.742927:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 55574528 -> 56623104 00002000:00010000:1.0:1789762836.742932:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 108553 -> 110601 00010000:00010000:1.0:1789762836.742944:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.742954:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.742961:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.742966:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762836.742969:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.742975:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.742990:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.742995:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046b7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.743069:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015037312:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14712us (100317us total) trans 4294967983 rc 0/0 00000100:00100000:1.0:1789762836.743078:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 779 00000100:00100000:1.0:1789762836.743128:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360032148822 time 360032233752 state 0x2 ref 0 00000100:00100000:2.0:1789762836.745956:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762836.746001:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015037440 00000100:00100000:1.0:1789762836.746013:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015037440 00000100:00100000:1.0:1789762836.746063:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360032148822 time 360032233752 state 0x0 ref 1 00000100:00100000:1.0:1789762836.746075:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360132233752 time 360032233752 state 0x1 ref 1 00000100:00100000:1.0:1789762836.828394:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360132233752 time 360132290007 state 0x0 ref 1 00000100:00100000:1.0:1789762836.828416:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015037440:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762836.828468:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 56623104, mtime: 0x6aad9d14, atime: 0x0, ctime: 0x6aad9d14, blocks: 0x1b009 00010000:00010000:1.0:1789762836.828477:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046be lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.828494:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=360s v=(0 0) 00010000:00010000:1.0:1789762836.828500:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.828505:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.828516:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762836.841943:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967984 is committed 00002000:00010000:2.0:1789762836.841965:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:2.0:1789762836.841971:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 110601 -> 112649 00010000:00010000:2.0:1789762836.841994:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046be lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762836.842005:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046be lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762836.842012:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762836.842017:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762836.842022:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046be lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762836.842031:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046be lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762836.842049:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046be lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762836.842055:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b046be lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762836.842183:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015037440:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13769us (96229us total) trans 4294967984 rc 0/0 00000100:00100000:2.0:1789762836.842194:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 780 00000100:00100000:2.0:1789762836.842205:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360132233752 time 360132290007 state 0x2 ref 0 00000100:00100000:2.0:1789762836.844412:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762836.844492:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015037568 00000100:00100000:0.0:1789762836.844503:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015037568 00000100:00100000:0.0:1789762836.844553:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360132233752 time 360132290007 state 0x0 ref 1 00000100:00100000:0.0:1789762836.844566:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360232290007 time 360132290007 state 0x1 ref 1 00000100:00100000:1.0:1789762836.928504:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360232290007 time 360232400239 state 0x0 ref 1 00000100:00100000:1.0:1789762836.928529:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015037568:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762836.928585:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 57671680, mtime: 0x6aad9d14, atime: 0x0, ctime: 0x6aad9d14, blocks: 0x1b809 00010000:00010000:1.0:1789762836.928595:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.928612:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=360s v=(0 0) 00010000:00010000:1.0:1789762836.928617:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762836.928623:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.928634:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762836.941931:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967985 is committed 00002000:00010000:1.0:1789762836.941979:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:1.0:1789762836.941984:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 1789762836 -> 1789762837 00002000:00010000:1.0:1789762836.941986:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 1789762836 -> 1789762837 00002000:00010000:1.0:1789762836.941988:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 112649 -> 114697 00010000:00010000:1.0:1789762836.941997:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.942006:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.942017:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.942022:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762836.942025:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.942031:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.942045:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762836.942050:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762836.942145:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015037568:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13624us (97733us total) trans 4294967985 rc 0/0 00000100:00100000:1.0:1789762836.942153:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 781 00000100:00100000:1.0:1789762836.942162:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360232290007 time 360232400239 state 0x2 ref 0 00000100:00100000:2.0:1789762836.944551:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762836.944612:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015037696 00000100:00100000:1.0:1789762836.944623:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015037696 00000100:00100000:1.0:1789762836.944669:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360232290007 time 360232400239 state 0x0 ref 1 00000100:00100000:1.0:1789762836.944689:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360332400239 time 360232400239 state 0x1 ref 1 00000100:00100000:1.0:1789762837.028602:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360332400239 time 360332491907 state 0x0 ref 1 00000100:00100000:1.0:1789762837.028633:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015037696:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762837.028685:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 58720256, mtime: 0x6aad9d15, atime: 0x0, ctime: 0x6aad9d15, blocks: 0x1c009 00010000:00010000:1.0:1789762837.028693:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046cc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.028707:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=360s v=(0 0) 00010000:00010000:1.0:1789762837.028711:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.028716:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.028727:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762837.042065:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967986 is committed 00002000:00010000:2.0:1789762837.042231:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:2.0:1789762837.042240:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 114697 -> 116745 00010000:00010000:2.0:1789762837.042255:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046cc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.042267:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046cc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.042275:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.042280:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762837.042284:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046cc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.042293:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046cc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.042312:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046cc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.042318:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b046cc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762837.042435:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015037696:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13810us (97885us total) trans 4294967986 rc 0/0 00000100:00100000:2.0:1789762837.042446:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 782 00000100:00100000:2.0:1789762837.042458:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360332400239 time 360332491907 state 0x2 ref 0 00000100:00100000:3.0:1789762837.044786:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762837.044867:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015037824 00000100:00100000:1.0:1789762837.044877:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015037824 00000100:00100000:1.0:1789762837.044922:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360332400239 time 360332491907 state 0x0 ref 1 00000100:00100000:1.0:1789762837.044934:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360432491907 time 360332491907 state 0x1 ref 1 00000004:00080000:1.0:1789762837.119478:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 20 committed 00000004:00080000:1.0:1789762837.119502:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 24 committed 00000100:00100000:0.0:1789762837.119593:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789762837.119595:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789762837.119629:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022173056:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1789762837.119629:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022173184:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789762837.119731:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762837.119780:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762837.119801:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022173184 00000100:00100000:3.0:1789762837.119812:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022173184 00000100:00100000:2.0:1789762837.119835:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022173056 00000100:00100000:2.0:1789762837.119842:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022173056 00000100:00080000:1.0:1789762837.119861:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789762837.119865:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762837.119894:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:2.0:1789762837.119910:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:3.0:1789762837.119911:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022173184:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762837.119924:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022173056:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1789762837.119939:0:6647:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:19 count 1 00002000:00080000:2.0:1789762837.119951:0:10916:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:13 count 1 00002000:00010000:3.0:1789762837.119970:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x13:0x0] initial LVB size: 8388608, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x4007 00010000:00010000:3.0:1789762837.119979:0:6647:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b046d3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762837.119981:0:10916:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xd:0x0] initial LVB size: 8388608, mtime: 0x6aad9d0e, atime: 0x0, ctime: 0x6aad9d0e, blocks: 0x4007 00010000:00010000:2.0:1789762837.119997:0:10916:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d7194a1f/0x416193e012b046da lrc: 3/0,1 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762837.120004:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=360s v=(0 0) 00010000:00010000:3.0:1789762837.120016:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762837.120020:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=360s v=(0 0) 00010000:00010000:2.0:1789762837.120025:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762837.120037:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b046d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.120038:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d7194a1f/0x416193e012b046da lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762837.120130:0:6647:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b046d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.120134:0:10916:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d7194a1f/0x416193e012b046da lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.120157:0:10916:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d7194a1f/0x416193e012b046da lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762837.120202:0:6647:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b046d3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.120242:0:10916:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d7194a1f/0x416193e012b046da lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762837.120243:0:6647:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b046d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.120297:0:10916:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:3.0:1789762837.120311:0:6647:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1789762837.120337:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1789762837.120342:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762837.120346:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d7194a1f/0x416193e012b046da lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.120350:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b046d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.120382:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762837.120386:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762837.120389:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d7194a1f/0x416193e012b046da lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.120393:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b046d3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.120440:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d7194a1f/0x416193e012b046da lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.120441:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b046d3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.120503:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b046d3 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.120506:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d7194a1f/0x416193e012b046da lrc: 1/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.120551:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b046d3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762837.120552:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d7194a1f/0x416193e012b046da lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.120610:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762837.120617:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762837.120620:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762837.120633:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762837.120639:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762837.120642:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762837.121214:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022173056:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1291us (1437us total) trans 4294967416 rc 0/0 00000100:00100000:2.0:1789762837.121222:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:0.0:1789762837.121237:0:3271:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967415 gen 2 00000004:00080000:0.0:1789762837.121245:0:3271:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981e902dd500, transno 4294967415 00000100:00100000:0.0:1789762837.121270:0:3271:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff981d8257d500 x1876702022173056/t4294967416(4294967416) o6->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789762853 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:0.0:1789762837.121281:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981d8257d500/2, rc 0, transno 4294967416 00000100:00100000:0.0:1789762837.121285:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022173056:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789762837.121381:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022173184:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1472us (1647us total) trans 4294967987 rc 0/0 00000004:00080000:0.0:1789762837.121389:0:3274:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebad01500/2, rc 0, transno 4294967987 00000100:00100000:3.0:1789762837.121396:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:0.0:1789762837.121398:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022173184:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762837.128677:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360432491907 time 360432571869 state 0x0 ref 1 00000100:00100000:2.0:1789762837.128700:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015037824:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762837.128769:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 59768832, mtime: 0x6aad9d15, atime: 0x0, ctime: 0x6aad9d15, blocks: 0x1c809 00010000:00010000:2.0:1789762837.128777:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b046e1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.128790:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=360s v=(0 0) 00010000:00010000:2.0:1789762837.128794:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762837.128799:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b046e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.128810:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b046e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762837.143449:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967988 is committed 00000001:00080000:0.0:1789762837.143469:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967987 is committed 00002000:00010000:1.0:1789762837.143507:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 59768832 -> 60817408 00002000:00010000:1.0:1789762837.143515:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 116745 -> 118793 00010000:00010000:1.0:1789762837.143526:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b046e1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.143536:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b046e1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.143543:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b046e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.143548:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762837.143552:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b046e1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.143558:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b046e1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.143573:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b046e1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.143578:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000075357237/0x416193e012b046e1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.143653:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015037824:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14957us (98868us total) trans 4294967988 rc 0/0 00000100:00100000:1.0:1789762837.143663:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 783 00000100:00100000:1.0:1789762837.143675:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360432491907 time 360432571869 state 0x2 ref 0 00000100:00100000:3.0:1789762837.146025:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762837.146127:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015037952 00000100:00100000:0.0:1789762837.146139:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015037952 00000100:00100000:0.0:1789762837.146196:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360432491907 time 360432571869 state 0x0 ref 1 00000100:00100000:0.0:1789762837.146211:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360532571869 time 360432571869 state 0x1 ref 1 00000100:00100000:1.0:1789762837.228826:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360532571869 time 360532721254 state 0x0 ref 1 00000100:00100000:1.0:1789762837.228850:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015037952:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762837.228898:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 60817408, mtime: 0x6aad9d15, atime: 0x0, ctime: 0x6aad9d15, blocks: 0x1d009 00010000:00010000:1.0:1789762837.228905:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046e8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.228919:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=360s v=(0 0) 00010000:00010000:1.0:1789762837.228923:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.228927:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.228938:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762837.242314:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967989 is committed 00002000:00010000:1.0:1789762837.242337:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 60817408 -> 61865984 00002000:00010000:1.0:1789762837.242343:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 118793 -> 120841 00010000:00010000:1.0:1789762837.242352:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046e8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.242360:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046e8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.242367:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.242373:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762837.242376:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046e8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.242382:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046e8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.242395:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046e8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.242400:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046e8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.242457:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015037952:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13615us (96431us total) trans 4294967989 rc 0/0 00000100:00100000:1.0:1789762837.242467:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 784 00000100:00100000:1.0:1789762837.242475:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360532571869 time 360532721254 state 0x2 ref 0 00000100:00100000:3.0:1789762837.244764:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762837.244831:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015038080 00000100:00100000:1.0:1789762837.244839:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015038080 00000100:00100000:1.0:1789762837.244874:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360532571869 time 360532721254 state 0x0 ref 1 00000100:00100000:1.0:1789762837.244885:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360632721254 time 360532721254 state 0x1 ref 1 00000100:00100000:2.0:1789762837.328923:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360632721254 time 360632813547 state 0x0 ref 1 00000100:00100000:2.0:1789762837.328970:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015038080:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762837.329060:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 61865984, mtime: 0x6aad9d15, atime: 0x0, ctime: 0x6aad9d15, blocks: 0x1d809 00010000:00010000:2.0:1789762837.329071:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b046ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.329131:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=360s v=(0 0) 00010000:00010000:2.0:1789762837.329136:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762837.329142:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b046ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762837.329157:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b046ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762837.342045:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967990 is committed 00002000:00010000:1.0:1789762837.342065:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 61865984 -> 62914560 00002000:00010000:1.0:1789762837.342072:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 120841 -> 122889 00010000:00010000:1.0:1789762837.342119:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b046ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.342134:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b046ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.342143:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b046ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.342149:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762837.342153:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b046ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.342159:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b046ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.342175:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b046ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.342180:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d1b571f/0x416193e012b046ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.342249:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015038080:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13285us (97483us total) trans 4294967990 rc 0/0 00000100:00100000:1.0:1789762837.342259:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 785 00000100:00100000:1.0:1789762837.342269:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360632721254 time 360632813547 state 0x2 ref 0 00000100:00100000:0.0:1789762837.344767:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762837.344806:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015038208 00000100:00100000:1.0:1789762837.344814:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015038208 00000100:00100000:1.0:1789762837.344853:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360632721254 time 360632813547 state 0x0 ref 1 00000100:00100000:1.0:1789762837.344864:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360732813547 time 360632813547 state 0x1 ref 1 00000100:00100000:1.0:1789762837.428975:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360732813547 time 360732870756 state 0x0 ref 1 00000100:00100000:1.0:1789762837.428998:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015038208:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762837.429046:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 62914560, mtime: 0x6aad9d15, atime: 0x0, ctime: 0x6aad9d15, blocks: 0x1e009 00010000:00010000:1.0:1789762837.429053:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046f6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.429067:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=360s v=(0 0) 00010000:00010000:1.0:1789762837.429070:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.429075:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.429122:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762837.431618:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762837.442323:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967991 is committed 00002000:00010000:1.0:1789762837.442358:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:1.0:1789762837.442364:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 122889 -> 124937 00010000:00010000:1.0:1789762837.442381:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046f6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.442393:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046f6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.442402:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.442408:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762837.442412:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046f6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.442419:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046f6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.442437:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046f6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.442442:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046f6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.442526:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015038208:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13533us (97756us total) trans 4294967991 rc 0/0 00000100:00100000:1.0:1789762837.442536:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 786 00000100:00100000:1.0:1789762837.442546:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360732813547 time 360732870756 state 0x2 ref 0 00000100:00100000:0.0:1789762837.444881:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762837.444965:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015038336 00000100:00100000:2.0:1789762837.444976:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015038336 00000100:00100000:2.0:1789762837.445027:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360732813547 time 360732870756 state 0x0 ref 1 00000100:00100000:2.0:1789762837.445039:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360832870756 time 360732870756 state 0x1 ref 1 00000100:00100000:1.0:1789762837.529121:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360832870756 time 360832981722 state 0x0 ref 1 00000100:00100000:1.0:1789762837.529151:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015038336:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762837.529215:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 63963136, mtime: 0x6aad9d15, atime: 0x0, ctime: 0x6aad9d15, blocks: 0x1e809 00010000:00010000:1.0:1789762837.529224:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046fd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.529241:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=360s v=(0 0) 00010000:00010000:1.0:1789762837.529246:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.529251:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.529265:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762837.543042:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967992 is committed 00002000:00010000:1.0:1789762837.543070:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 63963136 -> 65011712 00002000:00010000:1.0:1789762837.543109:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 124937 -> 126985 00010000:00010000:1.0:1789762837.543133:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046fd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.543149:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046fd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.543160:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.543170:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762837.543177:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046fd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.543190:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046fd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.543215:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046fd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.543225:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b046fd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.543338:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015038336:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14192us (98452us total) trans 4294967992 rc 0/0 00000100:00100000:1.0:1789762837.543354:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 787 00000100:00100000:1.0:1789762837.543370:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360832870756 time 360832981722 state 0x2 ref 0 00000100:00100000:0.0:1789762837.545795:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762837.545839:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015038464 00000100:00100000:1.0:1789762837.545850:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015038464 00000100:00100000:1.0:1789762837.545903:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360832870756 time 360832981722 state 0x0 ref 1 00000100:00100000:1.0:1789762837.545925:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360932981722 time 360832981722 state 0x1 ref 1 00000100:00100000:1.0:1789762837.629151:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360932981722 time 360933039382 state 0x0 ref 1 00000100:00100000:1.0:1789762837.629179:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015038464:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762837.629244:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 65011712, mtime: 0x6aad9d15, atime: 0x0, ctime: 0x6aad9d15, blocks: 0x1f009 00010000:00010000:1.0:1789762837.629253:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04704 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.629270:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=360s v=(0 0) 00010000:00010000:1.0:1789762837.629275:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.629280:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04704 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.629294:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04704 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762837.641894:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967993 is committed 00002000:00010000:1.0:1789762837.641912:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 65011712 -> 66060288 00002000:00010000:1.0:1789762837.641919:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 126985 -> 129033 00010000:00010000:1.0:1789762837.641934:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04704 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.641944:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04704 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.641952:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04704 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.641957:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762837.641962:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04704 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.641974:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04704 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.641991:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04704 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.641996:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04704 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.642070:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015038464:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12899us (96280us total) trans 4294967993 rc 0/0 00000100:00100000:1.0:1789762837.642130:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 788 00000100:00100000:1.0:1789762837.642141:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360932981722 time 360933039382 state 0x2 ref 0 00000100:00100000:0.0:1789762837.644264:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762837.644303:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015038592 00000100:00100000:1.0:1789762837.644313:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015038592 00000100:00100000:1.0:1789762837.644358:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 360932981722 time 360933039382 state 0x0 ref 1 00000100:00100000:1.0:1789762837.644370:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361033039382 time 360933039382 state 0x1 ref 1 00000100:00100000:1.0:1789762837.729196:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361033039382 time 361033088527 state 0x0 ref 1 00000100:00100000:1.0:1789762837.729223:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015038592:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762837.729288:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 66060288, mtime: 0x6aad9d15, atime: 0x0, ctime: 0x6aad9d15, blocks: 0x1f809 00010000:00010000:1.0:1789762837.729297:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0470b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.729314:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=361s v=(0 0) 00010000:00010000:1.0:1789762837.729318:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.729324:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0470b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.729340:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0470b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762837.744424:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967994 is committed 00002000:00010000:1.0:1789762837.744466:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 66060288 -> 67108864 00002000:00010000:1.0:1789762837.744472:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 129033 -> 131081 00010000:00010000:1.0:1789762837.744487:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0470b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.744499:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0470b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.744507:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0470b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.744512:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762837.744517:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0470b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.744530:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0470b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.744549:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0470b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.744553:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0470b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.744650:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015038592:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15432us (100386us total) trans 4294967994 rc 0/0 00000100:00100000:1.0:1789762837.744660:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 789 00000100:00100000:1.0:1789762837.744673:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361033039382 time 361033088527 state 0x2 ref 0 00000100:00100000:0.0:1789762837.746971:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762837.747052:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015038720 00000100:00100000:2.0:1789762837.747066:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015038720 00000100:00100000:2.0:1789762837.747173:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361033039382 time 361033088527 state 0x0 ref 1 00000100:00100000:2.0:1789762837.747189:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361133088527 time 361033088527 state 0x1 ref 1 00000100:00100000:1.0:1789762837.829293:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361133088527 time 361133181994 state 0x0 ref 1 00000100:00100000:1.0:1789762837.829320:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015038720:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762837.829385:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 67108864, mtime: 0x6aad9d15, atime: 0x0, ctime: 0x6aad9d15, blocks: 0x20009 00010000:00010000:1.0:1789762837.829394:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04712 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.829411:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=361s v=(0 0) 00010000:00010000:1.0:1789762837.829415:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.829421:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04712 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.829443:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04712 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762837.843506:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967995 is committed 00002000:00010000:1.0:1789762837.843546:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 67108864 -> 68157440 00002000:00010000:1.0:1789762837.843552:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 131081 -> 133129 00010000:00010000:1.0:1789762837.843564:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04712 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.843575:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04712 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.843581:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04712 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.843587:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762837.843591:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04712 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.843599:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04712 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.843617:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04712 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.843623:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04712 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.843702:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015038720:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14390us (96734us total) trans 4294967995 rc 0/0 00000100:00100000:1.0:1789762837.843711:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 790 00000100:00100000:1.0:1789762837.843720:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361133088527 time 361133181994 state 0x2 ref 0 00000100:00100000:0.0:1789762837.846013:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762837.846111:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015038848 00000100:00100000:1.0:1789762837.846122:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015038848 00000100:00100000:1.0:1789762837.846171:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361133088527 time 361133181994 state 0x0 ref 1 00000100:00100000:1.0:1789762837.846185:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361233181994 time 361133181994 state 0x1 ref 1 00000100:00100000:1.0:1789762837.929357:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361233181994 time 361233246667 state 0x0 ref 1 00000100:00100000:1.0:1789762837.929391:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015038848:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762837.929472:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 68157440, mtime: 0x6aad9d15, atime: 0x0, ctime: 0x6aad9d15, blocks: 0x20809 00010000:00010000:1.0:1789762837.929483:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04719 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.929503:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=361s v=(0 0) 00010000:00010000:1.0:1789762837.929509:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762837.929517:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04719 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.929534:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04719 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762837.945312:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967996 is committed 00002000:00010000:1.0:1789762837.945335:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 68157440 -> 69206016 00002000:00010000:1.0:1789762837.945340:0:14912:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 1789762837 -> 1789762838 00002000:00010000:1.0:1789762837.945343:0:14912:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 1789762837 -> 1789762838 00002000:00010000:1.0:1789762837.945345:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 133129 -> 135177 00010000:00010000:1.0:1789762837.945357:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04719 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.945367:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04719 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.945374:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04719 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.945378:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762837.945382:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04719 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.945388:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04719 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.945403:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04719 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762837.945409:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04719 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762837.945490:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015038848:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16107us (99478us total) trans 4294967996 rc 0/0 00000100:00100000:1.0:1789762837.945502:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 791 00000100:00100000:1.0:1789762837.945517:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361233181994 time 361233246667 state 0x2 ref 0 00000100:00100000:3.0:1789762837.947875:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762837.947944:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015038976 00000100:00100000:1.0:1789762837.947953:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015038976 00000100:00100000:1.0:1789762837.947994:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361233181994 time 361233246667 state 0x0 ref 1 00000100:00100000:1.0:1789762837.948006:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361333246667 time 361233246667 state 0x1 ref 1 00000100:00100000:1.0:1789762838.029430:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361333246667 time 361333325617 state 0x0 ref 1 00000100:00100000:1.0:1789762838.029475:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015038976:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:0.0:1789762838.029517:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789762838.029537:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 69206016, mtime: 0x6aad9d16, atime: 0x0, ctime: 0x6aad9d16, blocks: 0x21009 00010000:00010000:1.0:1789762838.029548:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04720 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.029566:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=361s v=(0 0) 00010000:00010000:1.0:1789762838.029571:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.029578:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04720 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.029602:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04720 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762838.042655:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967997 is committed 00002000:00010000:1.0:1789762838.042674:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 69206016 -> 70254592 00002000:00010000:1.0:1789762838.042680:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 135177 -> 137225 00010000:00010000:1.0:1789762838.042689:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04720 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.042697:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04720 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.042704:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04720 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.042709:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762838.042712:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04720 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.042719:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04720 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.042732:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04720 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.042737:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04720 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762838.042804:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839ea300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015038976:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13357us (94927us total) trans 4294967997 rc 0/0 00000100:00100000:1.0:1789762838.042814:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 792 00000100:00100000:1.0:1789762838.042822:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361333246667 time 361333325617 state 0x2 ref 0 00000100:00100000:2.0:1789762838.045181:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762838.045244:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015039104 00000100:00100000:0.0:1789762838.045252:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015039104 00000100:00100000:0.0:1789762838.045288:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361333246667 time 361333325617 state 0x0 ref 1 00000100:00100000:0.0:1789762838.045299:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361433325617 time 361333325617 state 0x1 ref 1 00000100:00100000:1.0:1789762838.129518:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361433325617 time 361433411047 state 0x0 ref 1 00000100:00100000:1.0:1789762838.129544:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015039104:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762838.129608:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 70254592, mtime: 0x6aad9d16, atime: 0x0, ctime: 0x6aad9d16, blocks: 0x21809 00010000:00010000:1.0:1789762838.129615:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04727 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.129629:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=361s v=(0 0) 00010000:00010000:1.0:1789762838.129634:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.129640:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04727 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.129656:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04727 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762838.144243:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967998 is committed 00002000:00010000:1.0:1789762838.144262:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 70254592 -> 71303168 00002000:00010000:1.0:1789762838.144267:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 137225 -> 139273 00010000:00010000:1.0:1789762838.144276:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04727 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.144285:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04727 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.144297:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04727 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.144302:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762838.144305:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04727 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.144311:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04727 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.144324:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04727 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.144329:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04727 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762838.144386:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015039104:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14852us (99212us total) trans 4294967998 rc 0/0 00000100:00100000:1.0:1789762838.144394:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 793 00000100:00100000:1.0:1789762838.144402:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361433325617 time 361433411047 state 0x2 ref 0 00000100:00100000:0.0:1789762838.146661:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762838.146706:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015039232 00000100:00100000:1.0:1789762838.146714:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015039232 00000100:00100000:1.0:1789762838.146749:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361433325617 time 361433411047 state 0x0 ref 1 00000100:00100000:1.0:1789762838.146763:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361533411047 time 361433411047 state 0x1 ref 1 00000100:00100000:1.0:1789762838.229583:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361533411047 time 361533479174 state 0x0 ref 1 00000100:00100000:1.0:1789762838.229606:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015039232:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762838.229657:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 71303168, mtime: 0x6aad9d16, atime: 0x0, ctime: 0x6aad9d16, blocks: 0x22009 00010000:00010000:1.0:1789762838.229665:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0472e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.229678:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=361s v=(0 0) 00010000:00010000:1.0:1789762838.229681:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.229686:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0472e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.229697:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0472e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762838.241129:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967999 is committed 00002000:00010000:1.0:1789762838.241134:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 71303168 -> 72351744 00002000:00010000:1.0:1789762838.241139:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 139273 -> 141321 00010000:00010000:1.0:1789762838.241149:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0472e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.241158:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0472e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.241168:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0472e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.241174:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762838.241177:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0472e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.241183:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0472e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.241196:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0472e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.241201:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0472e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762838.241257:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015039232:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11658us (94596us total) trans 4294967999 rc 0/0 00000100:00100000:1.0:1789762838.241266:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 794 00000100:00100000:1.0:1789762838.241274:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361533411047 time 361533479174 state 0x2 ref 0 00000100:00100000:0.0:1789762838.243326:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762838.243358:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015039360 00000100:00100000:1.0:1789762838.243366:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015039360 00000100:00100000:1.0:1789762838.243402:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361533411047 time 361533479174 state 0x0 ref 1 00000100:00100000:1.0:1789762838.243416:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361633479174 time 361533479174 state 0x1 ref 1 00000100:00100000:1.0:1789762838.329648:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361633479174 time 361633544766 state 0x0 ref 1 00000100:00100000:1.0:1789762838.329670:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015039360:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762838.329715:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 72351744, mtime: 0x6aad9d16, atime: 0x0, ctime: 0x6aad9d16, blocks: 0x22809 00010000:00010000:1.0:1789762838.329723:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04735 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.329736:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=361s v=(0 0) 00010000:00010000:1.0:1789762838.329740:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.329744:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04735 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.329755:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04735 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762838.341006:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968000 is committed 00002000:00010000:1.0:1789762838.341058:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 72351744 -> 73400320 00002000:00010000:1.0:1789762838.341063:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 141321 -> 143369 00010000:00010000:1.0:1789762838.341084:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04735 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.341109:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04735 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.341120:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04735 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.341125:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762838.341129:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04735 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.341134:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04735 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.341147:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04735 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.341152:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04735 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762838.341206:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015039360:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11543us (97881us total) trans 4294968000 rc 0/0 00000100:00100000:1.0:1789762838.341213:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 795 00000100:00100000:1.0:1789762838.341221:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361633479174 time 361633544766 state 0x2 ref 0 00000100:00100000:3.0:1789762838.343222:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762838.343298:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015039488 00000100:00100000:0.0:1789762838.343306:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015039488 00000100:00100000:0.0:1789762838.343341:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361633479174 time 361633544766 state 0x0 ref 1 00000100:00100000:0.0:1789762838.343351:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361733544766 time 361633544766 state 0x1 ref 1 00000100:00100000:1.0:1789762838.429765:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361733544766 time 361733659926 state 0x0 ref 1 00000100:00100000:1.0:1789762838.429789:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015039488:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762838.429841:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 73400320, mtime: 0x6aad9d16, atime: 0x0, ctime: 0x6aad9d16, blocks: 0x23009 00010000:00010000:1.0:1789762838.429849:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0473c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.429862:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=361s v=(0 0) 00010000:00010000:1.0:1789762838.429866:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.429871:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0473c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.429882:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0473c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762838.442652:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968001 is committed 00002000:00010000:1.0:1789762838.442718:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 73400320 -> 74448896 00002000:00010000:1.0:1789762838.442723:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 143369 -> 145417 00010000:00010000:1.0:1789762838.442734:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0473c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.442744:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0473c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.442751:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0473c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.442756:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762838.442761:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0473c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.442772:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0473c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.442787:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0473c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.442793:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0473c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762838.442868:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839ebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015039488:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13085us (99642us total) trans 4294968001 rc 0/0 00000100:00100000:1.0:1789762838.442877:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 796 00000100:00100000:1.0:1789762838.442886:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361733544766 time 361733659926 state 0x2 ref 0 00000100:00100000:3.0:1789762838.445209:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762838.445241:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015039616 00000100:00100000:1.0:1789762838.445249:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015039616 00000100:00100000:1.0:1789762838.445291:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361733544766 time 361733659926 state 0x0 ref 1 00000100:00100000:1.0:1789762838.445303:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361833659926 time 361733659926 state 0x1 ref 1 00010000:00010000:0.0:1789762838.455196:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762838.529822:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361833659926 time 361833717614 state 0x0 ref 1 00000100:00100000:1.0:1789762838.529843:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015039616:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762838.529892:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 74448896, mtime: 0x6aad9d16, atime: 0x0, ctime: 0x6aad9d16, blocks: 0x23809 00010000:00010000:1.0:1789762838.529900:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04743 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.529914:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=361s v=(0 0) 00010000:00010000:1.0:1789762838.529917:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.529922:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04743 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.529932:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04743 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762838.542190:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968002 is committed 00002000:00010000:1.0:1789762838.542254:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:1.0:1789762838.542259:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 145417 -> 147465 00010000:00010000:1.0:1789762838.542267:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04743 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.542276:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04743 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.542283:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04743 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.542287:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762838.542291:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04743 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.542298:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04743 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.542313:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04743 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.542318:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b04743 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762838.542373:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015039616:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12536us (97164us total) trans 4294968002 rc 0/0 00000100:00100000:1.0:1789762838.542381:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 797 00000100:00100000:1.0:1789762838.542389:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361833659926 time 361833717614 state 0x2 ref 0 00000100:00100000:3.0:1789762838.544426:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762838.544459:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015039744 00000100:00100000:1.0:1789762838.544467:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015039744 00000100:00100000:1.0:1789762838.544503:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361833659926 time 361833717614 state 0x0 ref 1 00000100:00100000:1.0:1789762838.544515:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361933717614 time 361833717614 state 0x1 ref 1 00000100:00100000:2.0:1789762838.626219:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762838.626250:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762838.626273:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762838.626306:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015039872 00000100:00100000:2.0:1789762838.626316:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762838.626319:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015040128 00000100:00100000:3.0:1789762838.626326:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015039872 00000100:00100000:0.0:1789762838.626327:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015040128 00000100:00100000:1.0:1789762838.626332:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015040000 00000100:00100000:1.0:1789762838.626339:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015040000 00000100:00100000:2.0:1789762838.626344:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015040256 00000100:00100000:2.0:1789762838.626349:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015040256 00000100:00100000:3.0:1789762838.626382:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 314 00000100:00100000:3.0:1789762838.626396:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85086850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015039872:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762838.626402:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 273 00000100:00100000:0.0:1789762838.626404:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 138 00000100:00100000:2.0:1789762838.626410:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 274 00000100:00100000:1.0:1789762838.626416:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702015040000:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762838.626421:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015040128:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762838.626425:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad02300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015040256:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762838.626467:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702015040000:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 52us (187us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762838.626468:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85086850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015039872:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 71us (251us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762838.626473:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015040128:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 55us (217us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762838.626476:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad02300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015040256:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 54us (159us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762838.626488:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 314 00000100:00100000:2.0:1789762838.626496:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 274 00000100:00100000:0.0:1789762838.626499:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 138 00000100:00100000:1.0:1789762838.626504:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 273 00000100:00080000:0.0:1789762838.626531:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762838.626538:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762838.629876:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361933717614 time 361933777189 state 0x0 ref 1 00000100:00100000:1.0:1789762838.629889:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d839e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015039744:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762838.629929:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 75497472, mtime: 0x6aad9d16, atime: 0x0, ctime: 0x6aad9d16, blocks: 0x24009 00010000:00010000:1.0:1789762838.629936:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0474a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.629949:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=361s v=(0 0) 00010000:00010000:1.0:1789762838.629953:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.629958:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0474a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.629969:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0474a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762838.641369:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 75497472 -> 76546048 00000001:00080000:3.0:1789762838.641372:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968003 is committed 00002000:00010000:2.0:1789762838.641375:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 147465 -> 149513 00010000:00010000:2.0:1789762838.641387:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0474a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.641397:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0474a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.641404:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0474a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.641420:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762838.641424:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0474a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.641430:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0474a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.641444:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0474a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.641450:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0474a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762838.641503:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d839e9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015039744:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11615us (97076us total) trans 4294968003 rc 0/0 00000100:00100000:2.0:1789762838.641511:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 798 00000100:00100000:2.0:1789762838.641520:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361933717614 time 361933777189 state 0x2 ref 0 00000100:00100000:0.0:1789762838.643780:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762838.643823:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015040384 00000100:00100000:2.0:1789762838.643829:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015040384 00000100:00100000:2.0:1789762838.643861:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 361933717614 time 361933777189 state 0x0 ref 1 00000100:00100000:2.0:1789762838.643871:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362033777189 time 361933777189 state 0x1 ref 1 00000100:00100000:0.0:1789762838.647239:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022173312:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762838.647287:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762838.647338:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022173312 00000100:00100000:3.0:1789762838.647346:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022173312 00000100:00100000:3.0:1789762838.647397:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:3.0:1789762838.647409:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022173312:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762838.647429:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3793747968 avail 3781165056 00000100:00100000:3.0:1789762838.647476:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022173312:12345-0@lo:13:osp-pre-0-0.0 Request processed in 70us (187us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762838.647484:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00100000:0.0:1789762838.647508:0:3274:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967987 gen 2 00000004:00080000:0.0:1789762838.647514:0:3274:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebad01500, transno 4294967987 00000100:00100000:0.0:1789762838.647533:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022173312:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762838.730000:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362033777189 time 362033895811 state 0x0 ref 1 00000100:00100000:1.0:1789762838.730023:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015040384:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762838.730112:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 76546048, mtime: 0x6aad9d16, atime: 0x0, ctime: 0x6aad9d16, blocks: 0x24809 00010000:00010000:1.0:1789762838.730121:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04751 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.730135:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=362s v=(0 0) 00010000:00010000:1.0:1789762838.730139:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.730143:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04751 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.730157:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04751 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762838.742331:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968004 is committed 00002000:00010000:3.0:1789762838.742356:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:3.0:1789762838.742362:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 149513 -> 151561 00010000:00010000:3.0:1789762838.742374:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04751 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.742384:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04751 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.742390:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04751 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.742395:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762838.742406:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04751 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.742413:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04751 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.742428:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04751 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762838.742433:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04751 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762838.742516:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015040384:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12496us (98733us total) trans 4294968004 rc 0/0 00000100:00100000:3.0:1789762838.742526:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 799 00000100:00100000:3.0:1789762838.742536:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362033777189 time 362033895811 state 0x2 ref 0 00000100:00100000:3.0:1789762838.745149:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762838.745202:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015040512 00000100:00100000:1.0:1789762838.745211:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015040512 00000100:00100000:1.0:1789762838.745251:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362033777189 time 362033895811 state 0x0 ref 1 00000100:00100000:1.0:1789762838.745262:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362133895811 time 362033895811 state 0x1 ref 1 00000100:00100000:1.0:1789762838.830137:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362133895811 time 362134027122 state 0x0 ref 1 00000100:00100000:1.0:1789762838.830174:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015040512:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762838.830227:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 77594624, mtime: 0x6aad9d16, atime: 0x0, ctime: 0x6aad9d16, blocks: 0x25009 00010000:00010000:1.0:1789762838.830234:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b04758 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.830248:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=362s v=(0 0) 00010000:00010000:1.0:1789762838.830252:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762838.830257:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b04758 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762838.830268:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b04758 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762838.844256:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968005 is committed 00002000:00010000:0.0:1789762838.844280:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:0.0:1789762838.844286:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 151561 -> 153609 00010000:00010000:0.0:1789762838.844302:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b04758 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.844314:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b04758 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.844322:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b04758 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.844327:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762838.844332:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b04758 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.844340:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b04758 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.844359:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b04758 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.844377:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b04758 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762838.844469:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015040512:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14308us (99317us total) trans 4294968005 rc 0/0 00000100:00100000:0.0:1789762838.844479:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 800 00000100:00100000:0.0:1789762838.844491:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362133895811 time 362134027122 state 0x2 ref 0 00000100:00100000:3.0:1789762838.846722:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762838.846802:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015040640 00000100:00100000:1.0:1789762838.846818:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015040640 00000100:00100000:1.0:1789762838.846893:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362133895811 time 362134027122 state 0x0 ref 1 00000100:00100000:1.0:1789762838.846908:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362234027122 time 362134027122 state 0x1 ref 1 00000100:00100000:2.0:1789762838.930233:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362234027122 time 362234128201 state 0x0 ref 1 00000100:00100000:2.0:1789762838.930269:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015040640:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762838.930317:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 78643200, mtime: 0x6aad9d16, atime: 0x0, ctime: 0x6aad9d16, blocks: 0x25809 00010000:00010000:2.0:1789762838.930324:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b0475f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.930338:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=362s v=(0 0) 00010000:00010000:2.0:1789762838.930342:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762838.930347:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b0475f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762838.930357:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b0475f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762838.945938:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968006 is committed 00002000:00010000:0.0:1789762838.945945:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:0.0:1789762838.945952:0:6654:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 1789762838 -> 1789762839 00002000:00010000:0.0:1789762838.945954:0:6654:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 1789762838 -> 1789762839 00002000:00010000:0.0:1789762838.945957:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 153609 -> 155657 00010000:00010000:0.0:1789762838.945968:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b0475f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.945978:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b0475f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.945985:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b0475f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.945991:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1a200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762838.945994:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b0475f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.946012:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b0475f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.946026:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b0475f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762838.946032:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7194a1f/0x416193e012b0475f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762838.946138:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015040640:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15871us (99413us total) trans 4294968006 rc 0/0 00000100:00100000:0.0:1789762838.946148:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 801 00000100:00100000:0.0:1789762838.946157:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362234027122 time 362234128201 state 0x2 ref 0 00000100:00100000:3.0:1789762838.948418:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762838.948466:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015040768 00000100:00100000:2.0:1789762838.948475:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015040768 00000100:00100000:2.0:1789762838.948526:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362234027122 time 362234128201 state 0x0 ref 1 00000100:00100000:2.0:1789762838.948540:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362334128201 time 362234128201 state 0x1 ref 1 00000100:00100000:3.0:1789762839.030364:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362334128201 time 362334252219 state 0x0 ref 1 00000100:00100000:3.0:1789762839.030394:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015040768:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762839.030464:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 79691776, mtime: 0x6aad9d17, atime: 0x0, ctime: 0x6aad9d17, blocks: 0x26009 00010000:00010000:3.0:1789762839.030492:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b04766 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.030510:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=362s v=(0 0) 00010000:00010000:3.0:1789762839.030514:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762839.030521:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b04766 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.030535:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b04766 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762839.045080:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:1.0:1789762839.045114:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 155657 -> 157705 00000001:00080000:0.0:1789762839.045125:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968007 is committed 00010000:00010000:1.0:1789762839.045128:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b04766 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.045141:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b04766 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.045148:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b04766 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.045155:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762839.045158:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b04766 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.045165:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b04766 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.045180:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b04766 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.045187:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b04766 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762839.045262:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015040768:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14878us (96845us total) trans 4294968007 rc 0/0 00000100:00100000:1.0:1789762839.045270:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 802 00000100:00100000:1.0:1789762839.045281:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362334128201 time 362334252219 state 0x2 ref 0 00000100:00100000:3.0:1789762839.047555:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762839.047601:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015040896 00000100:00100000:1.0:1789762839.047609:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015040896 00000100:00100000:1.0:1789762839.047650:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362334128201 time 362334252219 state 0x0 ref 1 00000100:00100000:1.0:1789762839.047661:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362434252219 time 362334252219 state 0x1 ref 1 00000100:00100000:0.0:1789762839.130478:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362434252219 time 362434369383 state 0x0 ref 1 00000100:00100000:0.0:1789762839.130512:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015040896:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762839.130571:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 80740352, mtime: 0x6aad9d17, atime: 0x0, ctime: 0x6aad9d17, blocks: 0x26809 00010000:00010000:0.0:1789762839.130581:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b0476d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.130600:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=362s v=(0 0) 00010000:00010000:0.0:1789762839.130605:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.130628:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b0476d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.130641:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b0476d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762839.144449:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968008 is committed 00002000:00010000:0.0:1789762839.144468:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 80740352 -> 81788928 00002000:00010000:0.0:1789762839.144474:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 157705 -> 159753 00010000:00010000:0.0:1789762839.144486:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b0476d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.144500:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b0476d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.144510:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b0476d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.144516:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbee00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762839.144519:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b0476d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.144526:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b0476d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.144543:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b0476d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.144567:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b0476d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762839.144645:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015040896:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14147us (97092us total) trans 4294968008 rc 0/0 00000100:00100000:0.0:1789762839.144654:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 803 00000100:00100000:0.0:1789762839.144663:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362434252219 time 362434369383 state 0x2 ref 0 00000100:00100000:3.0:1789762839.146779:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762839.146871:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015041024 00000100:00100000:1.0:1789762839.146880:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015041024 00000100:00100000:1.0:1789762839.146918:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362434252219 time 362434369383 state 0x0 ref 1 00000100:00100000:1.0:1789762839.146928:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362534369383 time 362434369383 state 0x1 ref 1 00000100:00100000:3.0:1789762839.159270:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022173440:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762839.159283:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022173824:0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.159286:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad00e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022173696:0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.159330:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad02a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022173952:0@lo:400:kworker.0 00000100:00100000:3.0:1789762839.159356:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762839.159364:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762839.159398:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022173824 00000100:00100000:1.0:1789762839.159407:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022173824 00000100:00100000:2.0:1789762839.159407:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762839.159460:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022173568:0@lo:400:kworker.0 00000100:00080000:0.0:1789762839.159480:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762839.159482:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022173952 00000100:00100000:1.0:1789762839.159485:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 275 00000100:00100000:2.0:1789762839.159488:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022173952 00000100:00100000:1.0:1789762839.159502:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022173824:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.159509:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762839.159523:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762839.159538:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 139 00000100:00100000:2.0:1789762839.159548:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022173952:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.159573:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc4f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022174208:0@lo:400:kworker.0 00000100:00100000:3.0:1789762839.159607:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad03800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022174080:0@lo:400:kworker.0 00000100:00100000:1.0:1789762839.159611:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022173824:12345-0@lo:400:kworker.0 Request processed in 111us (244us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762839.159626:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 275 00000100:00100000:2.0:1789762839.159627:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022173952:12345-0@lo:400:kworker.0 Request processed in 81us (222us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762839.159634:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762839.159647:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 139 00000100:00100000:3.0:1789762839.159660:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762839.159680:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022173696 00000100:00100000:2.0:1789762839.159682:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022174208 00000100:00100000:1.0:1789762839.159690:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022173696 00000100:00100000:2.0:1789762839.159691:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022174208 00000100:00100000:3.0:1789762839.159734:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022173440 00000100:00100000:0.0:1789762839.159743:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022173824:0@lo:400:kworker.0 00000100:00100000:3.0:1789762839.159744:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022173440 00000100:00100000:1.0:1789762839.159746:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022174080 00000100:00100000:2.0:1789762839.159754:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 140 00000100:00100000:1.0:1789762839.159758:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022174080 00000100:00100000:2.0:1789762839.159772:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3272:x1876702022174208:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.159803:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:1.0:1789762839.159815:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 277 00000100:00100000:0.0:1789762839.159824:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022173696:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762839.159830:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 116 00000100:00100000:1.0:1789762839.159838:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022174080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.159842:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3272:x1876702022174208:12345-0@lo:400:kworker.0 Request processed in 75us (210us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762839.159852:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3271:x1876702022173440:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762839.159853:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 140 00000100:00100000:0.0:1789762839.159869:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022173696:12345-0@lo:400:kworker.0 Request processed in 51us (361us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762839.159877:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00002000:00000024:3.0:1789762839.159908:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3851419648 avail 3849322496 00000100:00100000:2.0:1789762839.159909:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022173568 00000100:00100000:1.0:1789762839.159911:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022174080:12345-0@lo:400:kworker.0 Request processed in 83us (258us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762839.159919:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022173568 00000100:00100000:1.0:1789762839.159927:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 277 00000100:00100000:0.0:1789762839.159945:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad03800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022174080:0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.159977:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 315 00000100:00100000:2.0:1789762839.160007:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85085450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022173568:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762839.160007:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022173440:12345-0@lo:13:osp-pre-1-0.0 Request processed in 143us (638us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762839.160008:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762839.160010:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022173440:0@lo:13:osp-pre-1-0.0 00000100:00080000:1.0:1789762839.160022:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789762839.160033:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762839.160051:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 116 00000100:00100000:1.0:1789762839.160127:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc4f5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022174208:0@lo:400:kworker.0 00000100:00100000:2.0:1789762839.160135:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85085450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022173568:12345-0@lo:400:kworker.0 Request processed in 147us (621us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762839.160142:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 315 00000100:00100000:1.0:1789762839.160153:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad00e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022173696:0@lo:400:kworker.0 00000100:00100000:3.0:1789762839.160193:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad02a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022173952:0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.160241:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022173568:0@lo:400:kworker.0 00000100:00100000:0.0:1789762839.230621:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362534369383 time 362534509934 state 0x0 ref 1 00000100:00100000:0.0:1789762839.230665:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015041024:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762839.230751:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 81788928, mtime: 0x6aad9d17, atime: 0x0, ctime: 0x6aad9d17, blocks: 0x27009 00010000:00010000:0.0:1789762839.230764:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b04774 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.230787:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=362s v=(0 0) 00010000:00010000:0.0:1789762839.230791:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.230797:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b04774 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.230812:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b04774 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762839.245258:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968009 is committed 00002000:00010000:3.0:1789762839.245354:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:3.0:1789762839.245360:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 159753 -> 161801 00010000:00010000:3.0:1789762839.245374:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b04774 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.245387:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b04774 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.245398:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b04774 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.245407:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbee00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762839.245413:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b04774 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.245426:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b04774 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.245450:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b04774 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.245456:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aea6349e/0x416193e012b04774 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762839.245555:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015041024:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14905us (98780us total) trans 4294968009 rc 0/0 00000100:00100000:3.0:1789762839.245569:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 804 00000100:00100000:3.0:1789762839.245583:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362534369383 time 362534509934 state 0x2 ref 0 00000100:00100000:0.0:1789762839.247692:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762839.247745:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015041152 00000100:00100000:2.0:1789762839.247757:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015041152 00000100:00100000:2.0:1789762839.247809:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362534369383 time 362534509934 state 0x0 ref 1 00000100:00100000:2.0:1789762839.247823:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362634509934 time 362534509934 state 0x1 ref 1 00000100:00100000:1.0:1789762839.330745:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362634509934 time 362634628674 state 0x0 ref 1 00000100:00100000:1.0:1789762839.330782:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015041152:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762839.330849:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 82837504, mtime: 0x6aad9d17, atime: 0x0, ctime: 0x6aad9d17, blocks: 0x27809 00010000:00010000:1.0:1789762839.330859:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0477b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.330880:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=362s v=(0 0) 00010000:00010000:1.0:1789762839.330884:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762839.330890:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0477b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762839.330904:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0477b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762839.345531:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968010 is committed 00002000:00010000:3.0:1789762839.345585:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 82837504 -> 83886080 00002000:00010000:3.0:1789762839.345592:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 161801 -> 163849 00010000:00010000:3.0:1789762839.345604:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0477b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.345615:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0477b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.345622:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0477b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.345627:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762839.345631:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0477b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.345637:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0477b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.345652:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0477b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.345669:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0477b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762839.345763:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015041152:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14990us (98066us total) trans 4294968010 rc 0/0 00000100:00100000:3.0:1789762839.345775:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 805 00000100:00100000:3.0:1789762839.345789:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362634509934 time 362634628674 state 0x2 ref 0 00000100:00100000:0.0:1789762839.348116:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762839.348172:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015041280 00000100:00100000:1.0:1789762839.348183:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015041280 00000100:00100000:1.0:1789762839.348242:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362634509934 time 362634628674 state 0x0 ref 1 00000100:00100000:1.0:1789762839.348257:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362734628674 time 362634628674 state 0x1 ref 1 00000100:00100000:0.0:1789762839.430886:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362734628674 time 362734782354 state 0x0 ref 1 00000100:00100000:0.0:1789762839.430912:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015041280:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762839.430965:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 83886080, mtime: 0x6aad9d17, atime: 0x0, ctime: 0x6aad9d17, blocks: 0x28009 00010000:00010000:0.0:1789762839.430973:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04782 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.431000:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=362s v=(0 0) 00010000:00010000:0.0:1789762839.431004:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.431010:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04782 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.431021:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04782 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762839.443127:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968011 is committed 00002000:00010000:3.0:1789762839.443138:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 83886080 -> 84934656 00002000:00010000:3.0:1789762839.443143:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 163849 -> 165897 00010000:00010000:3.0:1789762839.443155:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04782 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.443167:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04782 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.443174:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04782 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.443179:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbfc00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762839.443183:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04782 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.443190:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04782 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.443219:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04782 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.443225:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b04782 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762839.443298:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015041280:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12394us (95182us total) trans 4294968011 rc 0/0 00000100:00100000:3.0:1789762839.443307:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 806 00000100:00100000:3.0:1789762839.443318:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362734628674 time 362734782354 state 0x2 ref 0 00000100:00100000:3.0:1789762839.445445:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762839.445491:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015041408 00000100:00100000:1.0:1789762839.445499:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015041408 00000100:00100000:1.0:1789762839.445539:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362734628674 time 362734782354 state 0x0 ref 1 00000100:00100000:1.0:1789762839.445550:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362834782354 time 362734782354 state 0x1 ref 1 00010000:00010000:1.0:1789762839.479264:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762839.530996:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362834782354 time 362834887599 state 0x0 ref 1 00000100:00100000:3.0:1789762839.531026:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015041408:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762839.531106:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 84934656, mtime: 0x6aad9d17, atime: 0x0, ctime: 0x6aad9d17, blocks: 0x28809 00010000:00010000:3.0:1789762839.531114:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04789 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.531128:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=362s v=(0 0) 00010000:00010000:3.0:1789762839.531132:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762839.531136:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04789 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.531163:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04789 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762839.542935:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968012 is committed 00002000:00010000:3.0:1789762839.542949:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:3.0:1789762839.542954:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 165897 -> 167945 00010000:00010000:3.0:1789762839.542965:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04789 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.542974:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04789 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.542981:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04789 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.542986:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762839.542990:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04789 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.542997:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04789 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.543010:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04789 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.543015:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04789 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762839.543120:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015041408:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12071us (97641us total) trans 4294968012 rc 0/0 00000100:00100000:3.0:1789762839.543133:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 807 00000100:00100000:3.0:1789762839.543144:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362834782354 time 362834887599 state 0x2 ref 0 00000100:00100000:3.0:1789762839.545243:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762839.545299:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015041536 00000100:00100000:2.0:1789762839.545307:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015041536 00000100:00100000:2.0:1789762839.545349:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362834782354 time 362834887599 state 0x0 ref 1 00000100:00100000:2.0:1789762839.545377:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362934887599 time 362834887599 state 0x1 ref 1 00000100:00100000:3.0:1789762839.631130:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362934887599 time 362935026583 state 0x0 ref 1 00000100:00100000:3.0:1789762839.631155:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015041536:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762839.631203:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 85983232, mtime: 0x6aad9d17, atime: 0x0, ctime: 0x6aad9d17, blocks: 0x29009 00010000:00010000:3.0:1789762839.631211:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04790 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.631224:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=362s v=(0 0) 00010000:00010000:3.0:1789762839.631227:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762839.631232:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04790 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.631243:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04790 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762839.643500:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968013 is committed 00002000:00010000:0.0:1789762839.643570:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 85983232 -> 87031808 00002000:00010000:0.0:1789762839.643575:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 167945 -> 169993 00010000:00010000:0.0:1789762839.643585:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04790 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.643594:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04790 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.643601:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04790 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.643606:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762839.643609:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04790 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.643616:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04790 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.643640:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04790 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.643646:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004393564a/0x416193e012b04790 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762839.643705:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015041536:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12557us (98460us total) trans 4294968013 rc 0/0 00000100:00100000:0.0:1789762839.643712:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 808 00000100:00100000:0.0:1789762839.643724:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362934887599 time 362935026583 state 0x2 ref 0 00000100:00100000:1.0:1789762839.646138:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762839.646184:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015041664 00000100:00100000:3.0:1789762839.646192:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015041664 00000100:00100000:3.0:1789762839.646236:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 362934887599 time 362935026583 state 0x0 ref 1 00000100:00100000:3.0:1789762839.646247:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363035026583 time 362935026583 state 0x1 ref 1 00000100:00100000:3.0:1789762839.731207:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363035026583 time 363035099213 state 0x0 ref 1 00000100:00100000:3.0:1789762839.731238:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015041664:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762839.731289:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 87031808, mtime: 0x6aad9d17, atime: 0x0, ctime: 0x6aad9d17, blocks: 0x29809 00010000:00010000:3.0:1789762839.731296:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04797 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.731310:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=363s v=(0 0) 00010000:00010000:3.0:1789762839.731313:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762839.731318:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04797 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.731331:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04797 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762839.742577:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968014 is committed 00002000:00010000:0.0:1789762839.742596:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 87031808 -> 88080384 00002000:00010000:0.0:1789762839.742601:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 169993 -> 172041 00010000:00010000:0.0:1789762839.742611:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04797 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.742620:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04797 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.742627:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04797 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.742632:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762839.742635:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04797 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.742653:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04797 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.742668:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04797 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.742673:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04797 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762839.742730:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015041664:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11504us (96591us total) trans 4294968014 rc 0/0 00000100:00100000:0.0:1789762839.742738:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 809 00000100:00100000:0.0:1789762839.742747:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363035026583 time 363035099213 state 0x2 ref 0 00000100:00100000:3.0:1789762839.744795:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762839.744873:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015041792 00000100:00100000:1.0:1789762839.744883:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015041792 00000100:00100000:1.0:1789762839.744928:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363035026583 time 363035099213 state 0x0 ref 1 00000100:00100000:1.0:1789762839.744940:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363135099213 time 363035099213 state 0x1 ref 1 00000100:00100000:3.0:1789762839.831323:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363135099213 time 363135217871 state 0x0 ref 1 00000100:00100000:3.0:1789762839.831348:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015041792:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762839.831396:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 88080384, mtime: 0x6aad9d17, atime: 0x0, ctime: 0x6aad9d17, blocks: 0x2a009 00010000:00010000:3.0:1789762839.831417:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b0479e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.831430:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=363s v=(0 0) 00010000:00010000:3.0:1789762839.831433:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762839.831438:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b0479e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762839.831450:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b0479e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762839.843035:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968015 is committed 00002000:00010000:0.0:1789762839.843074:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 88080384 -> 89128960 00002000:00010000:0.0:1789762839.843080:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 172041 -> 174089 00010000:00010000:0.0:1789762839.843109:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b0479e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.843119:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b0479e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.843126:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b0479e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.843141:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762839.843145:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b0479e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.843151:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b0479e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.843166:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b0479e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.843171:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b0479e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762839.843245:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015041792:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11904us (98447us total) trans 4294968015 rc 0/0 00000100:00100000:0.0:1789762839.843255:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 810 00000100:00100000:0.0:1789762839.843264:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363135099213 time 363135217871 state 0x2 ref 0 00000100:00100000:3.0:1789762839.845309:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762839.845352:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015041920 00000100:00100000:2.0:1789762839.845360:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015041920 00000100:00100000:2.0:1789762839.845399:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363135099213 time 363135217871 state 0x0 ref 1 00000100:00100000:2.0:1789762839.845411:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363235217871 time 363135217871 state 0x1 ref 1 00000100:00100000:0.0:1789762839.931422:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363235217871 time 363235317779 state 0x0 ref 1 00000100:00100000:0.0:1789762839.931453:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015041920:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762839.931502:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 89128960, mtime: 0x6aad9d17, atime: 0x0, ctime: 0x6aad9d17, blocks: 0x2a809 00010000:00010000:0.0:1789762839.931509:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.931541:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=363s v=(0 0) 00010000:00010000:0.0:1789762839.931544:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762839.931549:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.931561:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762839.942881:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968016 is committed 00002000:00010000:0.0:1789762839.942940:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 89128960 -> 90177536 00002000:00010000:0.0:1789762839.942945:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 1789762839 -> 1789762840 00002000:00010000:0.0:1789762839.942947:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 1789762839 -> 1789762840 00002000:00010000:0.0:1789762839.942949:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 174089 -> 176137 00010000:00010000:0.0:1789762839.942958:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.942967:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.942974:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.942979:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762839.942982:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.942998:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.943012:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762839.943017:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762839.943115:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015041920:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11676us (97805us total) trans 4294968016 rc 0/0 00000100:00100000:0.0:1789762839.943124:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 811 00000100:00100000:0.0:1789762839.943133:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363235217871 time 363235317779 state 0x2 ref 0 00000100:00100000:3.0:1789762839.945210:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762839.945252:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015042048 00000100:00100000:1.0:1789762839.945260:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015042048 00000100:00100000:1.0:1789762839.945298:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363235217871 time 363235317779 state 0x0 ref 1 00000100:00100000:1.0:1789762839.945310:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363335317779 time 363235317779 state 0x1 ref 1 00000100:00100000:0.0:1789762840.031549:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363335317779 time 363335439923 state 0x0 ref 1 00000100:00100000:0.0:1789762840.031603:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015042048:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:1.0:1789762840.031633:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789762840.031666:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 90177536, mtime: 0x6aad9d18, atime: 0x0, ctime: 0x6aad9d18, blocks: 0x2b009 00010000:00010000:0.0:1789762840.031676:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.031694:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=363s v=(0 0) 00010000:00010000:0.0:1789762840.031699:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.031706:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.031734:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762840.043636:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968017 is committed 00002000:00010000:0.0:1789762840.043703:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 90177536 -> 91226112 00002000:00010000:0.0:1789762840.043708:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 176137 -> 178185 00010000:00010000:0.0:1789762840.043718:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.043728:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.043734:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.043739:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762840.043742:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.043749:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.043764:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.043769:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762840.043849:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015042048:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12276us (98636us total) trans 4294968017 rc 0/0 00000100:00100000:0.0:1789762840.043860:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 812 00000100:00100000:0.0:1789762840.043870:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363335317779 time 363335439923 state 0x2 ref 0 00000100:00100000:3.0:1789762840.046144:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762840.046198:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015042176 00000100:00100000:1.0:1789762840.046206:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015042176 00000100:00100000:1.0:1789762840.046246:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363335317779 time 363335439923 state 0x0 ref 1 00000100:00100000:1.0:1789762840.046257:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363435439923 time 363335439923 state 0x1 ref 1 00000100:00100000:0.0:1789762840.131650:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363435439923 time 363435545500 state 0x0 ref 1 00000100:00100000:0.0:1789762840.131673:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015042176:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762840.131724:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 91226112, mtime: 0x6aad9d18, atime: 0x0, ctime: 0x6aad9d18, blocks: 0x2b809 00010000:00010000:0.0:1789762840.131732:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.131745:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=363s v=(0 0) 00010000:00010000:0.0:1789762840.131749:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.131753:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.131764:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762840.143483:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968018 is committed 00002000:00010000:0.0:1789762840.143502:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 91226112 -> 92274688 00002000:00010000:0.0:1789762840.143508:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 178185 -> 180233 00010000:00010000:0.0:1789762840.143518:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.143535:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.143542:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.143547:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762840.143551:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.143557:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047b3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.143570:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047b3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.143575:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762840.143637:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015042176:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11968us (97489us total) trans 4294968018 rc 0/0 00000100:00100000:0.0:1789762840.143648:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 813 00000100:00100000:0.0:1789762840.143659:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363435439923 time 363435545500 state 0x2 ref 0 00000100:00100000:3.0:1789762840.145889:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762840.145934:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015042304 00000100:00100000:2.0:1789762840.145942:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015042304 00000100:00100000:2.0:1789762840.145982:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363435439923 time 363435545500 state 0x0 ref 1 00000100:00100000:2.0:1789762840.145993:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363535545500 time 363435545500 state 0x1 ref 1 00000001:00080000:0.0:1789762840.195028:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967416 is committed 00000100:00100000:0.0:1789762840.231744:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363535545500 time 363535640315 state 0x0 ref 1 00000100:00100000:0.0:1789762840.231770:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015042304:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762840.231829:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 92274688, mtime: 0x6aad9d18, atime: 0x0, ctime: 0x6aad9d18, blocks: 0x2c009 00010000:00010000:0.0:1789762840.231836:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.231849:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=363s v=(0 0) 00010000:00010000:0.0:1789762840.231853:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.231857:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.231868:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762840.242862:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968019 is committed 00002000:00010000:0.0:1789762840.242871:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:0.0:1789762840.242876:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 180233 -> 182281 00010000:00010000:0.0:1789762840.242886:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.242896:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.242903:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.242918:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762840.242921:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.242928:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.242943:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.242948:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047ba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762840.243008:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015042304:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11246us (97119us total) trans 4294968019 rc 0/0 00000100:00100000:0.0:1789762840.243017:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 814 00000100:00100000:0.0:1789762840.243026:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363535545500 time 363535640315 state 0x2 ref 0 00000100:00100000:3.0:1789762840.245154:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762840.245204:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015042432 00000100:00100000:1.0:1789762840.245213:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015042432 00000100:00100000:1.0:1789762840.245254:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363535545500 time 363535640315 state 0x0 ref 1 00000100:00100000:1.0:1789762840.245269:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363635640315 time 363535640315 state 0x1 ref 1 00000100:00100000:0.0:1789762840.331852:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363635640315 time 363635747748 state 0x0 ref 1 00000100:00100000:0.0:1789762840.331875:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015042432:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762840.331927:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 93323264, mtime: 0x6aad9d18, atime: 0x0, ctime: 0x6aad9d18, blocks: 0x2c809 00010000:00010000:0.0:1789762840.331934:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047c1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.331959:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=363s v=(0 0) 00010000:00010000:0.0:1789762840.331963:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.331968:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.331979:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762840.343396:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968020 is committed 00002000:00010000:3.0:1789762840.343396:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:3.0:1789762840.343405:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 182281 -> 184329 00010000:00010000:3.0:1789762840.343418:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047c1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.343429:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047c1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.343447:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.343452:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbf800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762840.343456:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047c1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.343463:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047c1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.343478:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047c1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.343483:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008550d0a2/0x416193e012b047c1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762840.343545:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015042432:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11676us (98390us total) trans 4294968020 rc 0/0 00000100:00100000:3.0:1789762840.343554:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 815 00000100:00100000:3.0:1789762840.343563:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363635640315 time 363635747748 state 0x2 ref 0 00000100:00100000:3.0:1789762840.345746:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762840.345801:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015042560 00000100:00100000:1.0:1789762840.345811:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015042560 00000100:00100000:1.0:1789762840.345855:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363635640315 time 363635747748 state 0x0 ref 1 00000100:00100000:1.0:1789762840.345870:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363735747748 time 363635747748 state 0x1 ref 1 00000100:00100000:0.0:1789762840.431998:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363735747748 time 363735894281 state 0x0 ref 1 00000100:00100000:0.0:1789762840.432023:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015042560:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762840.432122:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 94371840, mtime: 0x6aad9d18, atime: 0x0, ctime: 0x6aad9d18, blocks: 0x2d009 00010000:00010000:0.0:1789762840.432133:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000037d9e58/0x416193e012b047c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.432150:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=363s v=(0 0) 00010000:00010000:0.0:1789762840.432155:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.432160:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000037d9e58/0x416193e012b047c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.432172:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000037d9e58/0x416193e012b047c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762840.443907:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968021 is committed 00002000:00010000:3.0:1789762840.443921:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 94371840 -> 95420416 00002000:00010000:3.0:1789762840.443926:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 184329 -> 186379 00010000:00010000:3.0:1789762840.443935:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000037d9e58/0x416193e012b047c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.443951:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000037d9e58/0x416193e012b047c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.443959:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000037d9e58/0x416193e012b047c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.443964:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbec00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762840.443967:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000037d9e58/0x416193e012b047c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.443973:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000037d9e58/0x416193e012b047c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.443987:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000037d9e58/0x416193e012b047c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.443992:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000037d9e58/0x416193e012b047c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762840.444045:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015042560:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12030us (98298us total) trans 4294968021 rc 0/0 00000100:00100000:3.0:1789762840.444054:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 816 00000100:00100000:3.0:1789762840.444080:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363735747748 time 363735894281 state 0x2 ref 0 00000100:00100000:3.0:1789762840.446366:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762840.446455:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015042688 00000100:00100000:2.0:1789762840.446464:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015042688 00000100:00100000:2.0:1789762840.446499:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363735747748 time 363735894281 state 0x0 ref 1 00000100:00100000:2.0:1789762840.446510:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363835894281 time 363735894281 state 0x1 ref 1 00010000:00010000:0.0:1789762840.503187:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762840.532145:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363835894281 time 363836039085 state 0x0 ref 1 00000100:00100000:0.0:1789762840.532176:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015042688:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762840.532230:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 95420416, mtime: 0x6aad9d18, atime: 0x0, ctime: 0x6aad9d18, blocks: 0x2d80b 00010000:00010000:0.0:1789762840.532237:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b047cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.532251:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=363s v=(0 0) 00010000:00010000:0.0:1789762840.532255:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762840.532260:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b047cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762840.532270:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b047cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762840.543665:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 95420416 -> 96468992 00000001:00080000:2.0:1789762840.543668:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968022 is committed 00002000:00010000:1.0:1789762840.543671:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 186379 -> 188427 00010000:00010000:1.0:1789762840.543683:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b047cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.543693:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b047cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.543700:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b047cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.543705:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762840.543709:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b047cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.543715:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b047cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.543729:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b047cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.543734:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004b9f9a09/0x416193e012b047cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762840.543809:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015042688:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11647us (97443us total) trans 4294968022 rc 0/0 00000100:00100000:1.0:1789762840.543819:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 817 00000100:00100000:1.0:1789762840.543829:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363835894281 time 363836039085 state 0x2 ref 0 00000100:00100000:3.0:1789762840.546169:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762840.546203:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015042816 00000100:00100000:1.0:1789762840.546212:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015042816 00000100:00100000:1.0:1789762840.546249:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363835894281 time 363836039085 state 0x0 ref 1 00000100:00100000:1.0:1789762840.546261:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363936039085 time 363836039085 state 0x1 ref 1 00000100:00100000:3.0:1789762840.632264:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363936039085 time 363936159498 state 0x0 ref 1 00000100:00100000:3.0:1789762840.632290:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015042816:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762840.632343:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 96468992, mtime: 0x6aad9d18, atime: 0x0, ctime: 0x6aad9d18, blocks: 0x2e00b 00010000:00010000:3.0:1789762840.632350:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.632364:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=363s v=(0 0) 00010000:00010000:3.0:1789762840.632367:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762840.632372:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.632383:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762840.643932:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968023 is committed 00002000:00010000:3.0:1789762840.643962:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 96468992 -> 97517568 00002000:00010000:3.0:1789762840.643968:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 188427 -> 190475 00010000:00010000:3.0:1789762840.643980:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.643994:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.644004:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.644027:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762840.644032:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.644041:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.644074:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.644083:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762840.644176:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015042816:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11893us (98004us total) trans 4294968023 rc 0/0 00000100:00100000:3.0:1789762840.644187:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 818 00000100:00100000:3.0:1789762840.644199:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363936039085 time 363936159498 state 0x2 ref 0 00000100:00100000:2.0:1789762840.646535:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762840.646588:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015042944 00000100:00100000:1.0:1789762840.646597:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015042944 00000100:00100000:1.0:1789762840.646637:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 363936039085 time 363936159498 state 0x0 ref 1 00000100:00100000:1.0:1789762840.646649:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364036159498 time 363936159498 state 0x1 ref 1 00000100:00100000:3.0:1789762840.732350:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364036159498 time 364036246770 state 0x0 ref 1 00000100:00100000:3.0:1789762840.732380:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015042944:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762840.732437:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 97517568, mtime: 0x6aad9d18, atime: 0x0, ctime: 0x6aad9d18, blocks: 0x2e80b 00010000:00010000:3.0:1789762840.732445:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.732472:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=364s v=(0 0) 00010000:00010000:3.0:1789762840.732475:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762840.732480:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.732493:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762840.744233:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968024 is committed 00002000:00010000:3.0:1789762840.744267:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 97517568 -> 98566144 00002000:00010000:3.0:1789762840.744273:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 190475 -> 192523 00010000:00010000:3.0:1789762840.744286:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.744300:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.744312:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.744322:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762840.744328:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.744339:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.744374:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.744384:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762840.744460:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015042944:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12089us (97924us total) trans 4294968024 rc 0/0 00000100:00100000:3.0:1789762840.744473:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 819 00000100:00100000:3.0:1789762840.744485:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364036159498 time 364036246770 state 0x2 ref 0 00000100:00100000:0.0:1789762840.746696:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762840.746770:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015043072 00000100:00100000:2.0:1789762840.746784:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015043072 00000100:00100000:2.0:1789762840.746822:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364036159498 time 364036246770 state 0x0 ref 1 00000100:00100000:2.0:1789762840.746833:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364136246770 time 364036246770 state 0x1 ref 1 00000100:00100000:1.0:1789762840.832433:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364136246770 time 364136329830 state 0x0 ref 1 00000100:00100000:1.0:1789762840.832460:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015043072:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762840.832509:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 98566144, mtime: 0x6aad9d18, atime: 0x0, ctime: 0x6aad9d18, blocks: 0x2f00b 00010000:00010000:1.0:1789762840.832517:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b047e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.832535:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=364s v=(0 0) 00010000:00010000:1.0:1789762840.832539:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762840.832545:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b047e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762840.832559:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b047e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762840.843978:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968025 is committed 00002000:00010000:3.0:1789762840.844035:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 98566144 -> 99614720 00002000:00010000:3.0:1789762840.844041:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 192523 -> 194571 00010000:00010000:3.0:1789762840.844050:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b047e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.844072:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b047e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.844080:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b047e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.844100:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762840.844104:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b047e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.844124:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b047e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.844138:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b047e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.844146:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b047e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762840.844215:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015043072:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11764us (97517us total) trans 4294968025 rc 0/0 00000100:00100000:3.0:1789762840.844223:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 820 00000100:00100000:3.0:1789762840.844232:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364136246770 time 364136329830 state 0x2 ref 0 00000100:00100000:0.0:1789762840.846391:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762840.846466:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015043200 00000100:00100000:1.0:1789762840.846475:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015043200 00000100:00100000:1.0:1789762840.846510:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364136246770 time 364136329830 state 0x0 ref 1 00000100:00100000:1.0:1789762840.846522:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364236329830 time 364136329830 state 0x1 ref 1 00000100:00100000:3.0:1789762840.932506:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364236329830 time 364236402480 state 0x0 ref 1 00000100:00100000:3.0:1789762840.932532:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015043200:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762840.932583:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 99614720, mtime: 0x6aad9d18, atime: 0x0, ctime: 0x6aad9d18, blocks: 0x2f80b 00010000:00010000:3.0:1789762840.932591:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.932604:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=364s v=(0 0) 00010000:00010000:3.0:1789762840.932607:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762840.932612:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.932635:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762840.943901:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968026 is committed 00002000:00010000:3.0:1789762840.943953:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 99614720 -> 100663296 00002000:00010000:3.0:1789762840.943957:0:14912:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime from disk: 1789762840 -> 1789762841 00002000:00010000:3.0:1789762840.943960:0:14912:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime from disk: 1789762840 -> 1789762841 00002000:00010000:3.0:1789762840.943961:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 194571 -> 196619 00010000:00010000:3.0:1789762840.943972:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.943982:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.943989:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.943994:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762840.943998:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.944004:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.944019:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762840.944036:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762840.944154:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015043200:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11628us (97760us total) trans 4294968026 rc 0/0 00000100:00100000:3.0:1789762840.944163:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 821 00000100:00100000:3.0:1789762840.944172:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364236329830 time 364236402480 state 0x2 ref 0 00000100:00100000:0.0:1789762840.946298:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762840.946359:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015043328 00000100:00100000:1.0:1789762840.946368:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015043328 00000100:00100000:1.0:1789762840.946420:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364236329830 time 364236402480 state 0x0 ref 1 00000100:00100000:1.0:1789762840.946437:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364336402480 time 364236402480 state 0x1 ref 1 00000100:00100000:3.0:1789762841.032603:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364336402480 time 364336497955 state 0x0 ref 1 00000100:00100000:3.0:1789762841.032628:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015043328:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762841.032678:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 100663296, mtime: 0x6aad9d19, atime: 0x0, ctime: 0x6aad9d19, blocks: 0x3000b 00010000:00010000:3.0:1789762841.032686:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.032698:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=364s v=(0 0) 00010000:00010000:3.0:1789762841.032702:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.032706:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.032718:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762841.045481:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968027 is committed 00002000:00010000:3.0:1789762841.045498:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 100663296 -> 101711872 00002000:00010000:3.0:1789762841.045513:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 196619 -> 198667 00010000:00010000:3.0:1789762841.045528:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.045543:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.045554:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.045561:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762841.045565:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.045572:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.045589:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.045594:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762841.045671:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015043328:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13050us (99372us total) trans 4294968027 rc 0/0 00000100:00100000:3.0:1789762841.045680:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 822 00000100:00100000:3.0:1789762841.045690:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364336402480 time 364336497955 state 0x2 ref 0 00000100:00100000:0.0:1789762841.047791:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762841.047844:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015043456 00000100:00100000:2.0:1789762841.047853:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015043456 00000100:00100000:2.0:1789762841.047894:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364336402480 time 364336497955 state 0x0 ref 1 00000100:00100000:2.0:1789762841.047906:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364436497955 time 364336497955 state 0x1 ref 1 00000100:00100000:3.0:1789762841.132734:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364436497955 time 364436628961 state 0x0 ref 1 00000100:00100000:3.0:1789762841.132760:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015043456:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762841.132812:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 101711872, mtime: 0x6aad9d19, atime: 0x0, ctime: 0x6aad9d19, blocks: 0x3080b 00010000:00010000:3.0:1789762841.132819:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.132833:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=364s v=(0 0) 00010000:00010000:3.0:1789762841.132837:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.132842:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.132853:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762841.144873:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968028 is committed 00002000:00010000:3.0:1789762841.144927:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:3.0:1789762841.144932:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 198667 -> 200715 00010000:00010000:3.0:1789762841.144943:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.144952:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.144959:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.144977:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762841.144980:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.144987:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.145001:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.145006:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b047f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762841.145114:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015043456:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12335us (97296us total) trans 4294968028 rc 0/0 00000100:00100000:3.0:1789762841.145124:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 823 00000100:00100000:3.0:1789762841.145137:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364436497955 time 364436628961 state 0x2 ref 0 00000100:00100000:0.0:1789762841.147278:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762841.147328:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015043584 00000100:00100000:1.0:1789762841.147336:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015043584 00000100:00100000:1.0:1789762841.147375:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364436497955 time 364436628961 state 0x0 ref 1 00000100:00100000:1.0:1789762841.147390:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364536628961 time 364436628961 state 0x1 ref 1 00000100:00100000:3.0:1789762841.232840:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364536628961 time 364536736525 state 0x0 ref 1 00000100:00100000:3.0:1789762841.232865:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015043584:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762841.232912:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 102760448, mtime: 0x6aad9d19, atime: 0x0, ctime: 0x6aad9d19, blocks: 0x3100b 00010000:00010000:3.0:1789762841.232921:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04800 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.232935:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=364s v=(0 0) 00010000:00010000:3.0:1789762841.232939:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.232943:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04800 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.232955:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04800 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762841.245356:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968029 is committed 00002000:00010000:3.0:1789762841.245366:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:3.0:1789762841.245371:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 200715 -> 202763 00010000:00010000:3.0:1789762841.245381:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04800 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.245392:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04800 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.245398:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04800 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.245404:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762841.245407:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04800 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.245414:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04800 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.245434:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04800 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.245441:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04800 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762841.245513:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015043584:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12653us (98233us total) trans 4294968029 rc 0/0 00000100:00100000:3.0:1789762841.245524:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 824 00000100:00100000:3.0:1789762841.245538:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364536628961 time 364536736525 state 0x2 ref 0 00000100:00100000:1.0:1789762841.247601:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762841.247647:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015043712 00000100:00100000:2.0:1789762841.247658:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015043712 00000100:00100000:2.0:1789762841.247705:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364536628961 time 364536736525 state 0x0 ref 1 00000100:00100000:2.0:1789762841.247720:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364636736525 time 364536736525 state 0x1 ref 1 00000100:00100000:3.0:1789762841.332948:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364636736525 time 364636843383 state 0x0 ref 1 00000100:00100000:3.0:1789762841.332975:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015043712:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762841.333025:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 103809024, mtime: 0x6aad9d19, atime: 0x0, ctime: 0x6aad9d19, blocks: 0x3180b 00010000:00010000:3.0:1789762841.333033:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04807 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.333046:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=364s v=(0 0) 00010000:00010000:3.0:1789762841.333063:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.333068:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04807 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.333081:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04807 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762841.344795:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968030 is committed 00002000:00010000:3.0:1789762841.344809:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size from disk: 103809024 -> 104857600 00002000:00010000:3.0:1789762841.344814:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb blocks from disk: 202763 -> 204811 00010000:00010000:3.0:1789762841.344827:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04807 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.344837:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04807 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.344844:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04807 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.344849:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762841.344853:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04807 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.344859:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04807 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.344874:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04807 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.344880:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04807 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762841.344949:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015043712:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11983us (97345us total) trans 4294968030 rc 0/0 00000100:00100000:3.0:1789762841.344958:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 825 00000100:00100000:3.0:1789762841.344968:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364636736525 time 364636843383 state 0x2 ref 0 00000100:00100000:0.0:1789762841.346284:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762841.346362:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015043968 00000100:00100000:1.0:1789762841.346371:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015043968 00000100:00100000:1.0:1789762841.346418:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 94 00000100:00100000:1.0:1789762841.346428:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b1a450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:14298:x1876702015043968:12345-192.168.203.48@tcp:35:dd.500 00000100:00080000:0.0:1789762841.346430:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:1.0:1789762841.346616:0:5807:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967552 00000100:00100000:1.0:1789762841.346677:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b1a450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14298:x1876702015043968:12345-192.168.203.48@tcp:35:dd.500 Request processed in 251us (396us total) trans 4294967553 rc 0/0 00000100:00100000:1.0:1789762841.346684:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 94 00000020:00080000:1.0:1789762841.346687:0:5807:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:1.0:1789762841.393321:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762841.393391:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015044224 00000100:00100000:2.0:1789762841.393402:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015044224 00000100:00100000:2.0:1789762841.393458:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 301 00000100:00100000:2.0:1789762841.393470:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14315:x1876702015044224:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:2.0:1789762841.393495:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762841.393518:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0480e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6b7 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.393617:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b04815 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.393648:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b04815 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.393663:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b04815 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762841.393676:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 104857600 00010000:00010000:2.0:1789762841.393800:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1ae00/0x416193e012b0480e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6b7 expref: 13 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.393814:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b04815 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6b7 expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.393821:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff981ea0aa3500) 00010000:00010000:2.0:1789762841.393823:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:2.0:1789762841.393826:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87c1a800, rc 0) 00000100:00100000:2.0:1789762841.393876:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14315:x1876702015044224:12345-192.168.203.48@tcp:101:dd.500 Request processed in 410us (563us total) trans 4294967554 rc 0/0 00000100:00100000:2.0:1789762841.393881:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 301 00000100:00100000:3.0:1789762841.395204:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762841.395239:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015044352 00000100:00100000:2.0:1789762841.395245:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015044352 00000100:00100000:2.0:1789762841.395276:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 182 00000100:00100000:2.0:1789762841.395281:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015044352:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:2.0:1789762841.395289:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762841.395300:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04823 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6be expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762841.395330:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 104857600, mtime: 0x6aad9d19, atime: 0x0, ctime: 0x6aad9d19, blocks: 0x3200b 00010000:00010000:2.0:1789762841.395343:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=364s v=(0 0) 00010000:00010000:2.0:1789762841.395347:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762841.395354:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04823 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6be expref: 6 pid: 6881 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.395364:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04823 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6be expref: 6 pid: 6881 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.395373:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87c1ba00, rc 0) 00000100:00100000:2.0:1789762841.395401:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015044352:12345-192.168.203.48@tcp:101:dd.500 Request processed in 119us (197us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762841.395409:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 182 00000100:00100000:0.0:1789762841.397643:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762841.397680:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015044480 00000100:00100000:2.0:1789762841.397684:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015044480 00000100:00100000:2.0:1789762841.397714:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333630663195 time 333630778521 state 0x0 ref 1 00000100:00100000:2.0:1789762841.397721:0:14913:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 8: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981ebc5ded00 gen 0 rate 20 depth 3 token 0 deadline 333630663195 time 333630778521 state 0x0 ref 1 00000100:00100000:2.0:1789762841.397746:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 3 deadline 364801653244 time 364701653244 state 0x1 ref 1 00000100:00100000:2.0:1789762841.397769:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 364801653244 time 364701675655 state 0x0 ref 1 00000100:00100000:2.0:1789762841.397793:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015044480:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762841.397813:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0482a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.397822:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762841.397825:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762841.397828:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0482a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.397835:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0482a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.401342:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0482a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.401353:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0482a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.401360:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0482a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.401366:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762841.401369:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0482a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.401375:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0482a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.401396:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0482a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.401402:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000caa49398/0x416193e012b0482a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762841.401441:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015044480:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3648us (3797us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762841.401450:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 826 00000100:00100000:2.0:1789762841.401460:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 364801653244 time 364701675655 state 0x2 ref 0 00000100:00100000:0.0:1789762841.402901:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762841.402940:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015044608 00000100:00100000:3.0:1789762841.402946:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015044608 00000100:00100000:3.0:1789762841.402970:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 364801653244 time 364701675655 state 0x0 ref 1 00000100:00100000:3.0:1789762841.402979:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 364801675655 time 364701675655 state 0x1 ref 1 00000100:00100000:3.0:1789762841.402998:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 364801675655 time 364706907326 state 0x0 ref 1 00000100:00100000:3.0:1789762841.403005:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82086680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015044608:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762841.403019:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000049244496/0x416193e012b04831 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.403028:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.403031:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.403034:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000049244496/0x416193e012b04831 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.403042:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000049244496/0x416193e012b04831 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.405579:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000049244496/0x416193e012b04831 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.405591:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000049244496/0x416193e012b04831 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.405598:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000049244496/0x416193e012b04831 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.405603:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9063a600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762841.405605:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000049244496/0x416193e012b04831 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.405612:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000049244496/0x416193e012b04831 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.405621:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000049244496/0x416193e012b04831 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.405626:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000049244496/0x416193e012b04831 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762841.405665:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82086680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015044608:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2658us (2764us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762841.405671:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 827 00000100:00100000:1.0:1789762841.405678:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 364801675655 time 364706907326 state 0x2 ref 0 00000100:00100000:0.0:1789762841.407266:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762841.407298:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015044736 00000100:00100000:2.0:1789762841.407305:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015044736 00000100:00100000:2.0:1789762841.407337:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 364801675655 time 364706907326 state 0x0 ref 1 00000100:00100000:2.0:1789762841.407351:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 364806907326 time 364706907326 state 0x1 ref 1 00000100:00100000:2.0:1789762841.407380:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364806907326 time 364711286289 state 0x0 ref 1 00000100:00100000:2.0:1789762841.407394:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82084700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015044736:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762841.407413:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04838 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.407423:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762841.407432:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762841.407436:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04838 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.407445:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04838 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.410234:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04838 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.410249:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04838 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.410259:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04838 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.410268:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ae00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762841.410273:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04838 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.410284:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04838 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.410299:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04838 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.410305:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b04838 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762841.410357:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82084700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015044736:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2966us (3091us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762841.410365:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 828 00000100:00100000:2.0:1789762841.410372:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364806907326 time 364711286289 state 0x2 ref 0 00000100:00100000:0.0:1789762841.412028:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762841.412105:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015044864 00000100:00100000:2.0:1789762841.412112:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015044864 00000100:00100000:2.0:1789762841.412147:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364806907326 time 364711286289 state 0x0 ref 1 00000100:00100000:2.0:1789762841.412158:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364811286289 time 364711286289 state 0x1 ref 1 00000100:00100000:3.0:1789762841.507524:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364811286289 time 364811418613 state 0x0 ref 1 00000100:00100000:3.0:1789762841.507550:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015044864:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762841.507587:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0483f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.507600:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.507604:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.507608:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0483f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.507619:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0483f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.510450:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0483f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.510463:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0483f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.510473:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0483f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.510482:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762841.510487:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0483f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.510499:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0483f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.510519:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0483f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.510529:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b0483f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762841.510589:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015044864:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3049us (98561us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762841.510601:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 829 00000100:00100000:3.0:1789762841.510611:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364811286289 time 364811418613 state 0x2 ref 0 00000100:00100000:3.0:1789762841.512675:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762841.512723:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015044992 00000100:00100000:1.0:1789762841.512732:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015044992 00000100:00100000:1.0:1789762841.512769:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364811286289 time 364811418613 state 0x0 ref 1 00000100:00100000:1.0:1789762841.512780:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364911418613 time 364811418613 state 0x1 ref 1 00010000:00010000:3.0:1789762841.527227:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762841.607583:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364911418613 time 364911475703 state 0x0 ref 1 00000100:00100000:1.0:1789762841.607611:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015044992:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762841.607649:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b04846 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.607664:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762841.607668:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762841.607672:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b04846 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.607683:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b04846 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.611164:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b04846 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.611183:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b04846 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.611192:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b04846 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.611198:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762841.611204:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b04846 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.611212:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b04846 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.611231:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b04846 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.611242:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b04846 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762841.611321:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015044992:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3718us (98643us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762841.611330:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 830 00000100:00100000:1.0:1789762841.611342:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364911418613 time 364911475703 state 0x2 ref 0 00000100:00100000:0.0:1789762841.613254:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762841.613294:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015045120 00000100:00100000:1.0:1789762841.613304:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015045120 00000100:00100000:1.0:1789762841.613353:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364911418613 time 364911475703 state 0x0 ref 1 00000100:00100000:1.0:1789762841.613366:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365011475703 time 364911475703 state 0x1 ref 1 00000100:00100000:2.0:1789762841.707681:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365011475703 time 365011577068 state 0x0 ref 1 00000100:00100000:2.0:1789762841.707707:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82087480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015045120:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762841.707744:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0484d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.707759:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762841.707762:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762841.707767:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0484d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762841.707779:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0484d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.711174:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0484d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.711191:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0484d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.711200:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0484d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.711206:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c1ae00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762841.711211:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0484d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.711220:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0484d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.711239:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0484d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.711245:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c314097b/0x416193e012b0484d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762841.711342:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82087480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015045120:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3641us (98088us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762841.711356:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 831 00000100:00100000:1.0:1789762841.711371:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365011475703 time 365011577068 state 0x2 ref 0 00000100:00100000:0.0:1789762841.713225:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762841.713278:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015045248 00000100:00100000:2.0:1789762841.713287:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015045248 00000100:00100000:2.0:1789762841.713329:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365011475703 time 365011577068 state 0x0 ref 1 00000100:00100000:2.0:1789762841.713340:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365111577068 time 365011577068 state 0x1 ref 1 00000100:00100000:3.0:1789762841.807763:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365111577068 time 365111660694 state 0x0 ref 1 00000100:00100000:3.0:1789762841.807789:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82086300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015045248:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762841.807831:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021f8c8e4/0x416193e012b04854 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.807847:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.807851:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762841.807855:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000021f8c8e4/0x416193e012b04854 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.807867:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000021f8c8e4/0x416193e012b04854 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.814296:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000021f8c8e4/0x416193e012b04854 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.814308:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000021f8c8e4/0x416193e012b04854 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.814315:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000021f8c8e4/0x416193e012b04854 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.814326:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81683000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762841.814330:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000021f8c8e4/0x416193e012b04854 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.814338:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000021f8c8e4/0x416193e012b04854 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.814357:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000021f8c8e4/0x416193e012b04854 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762841.814363:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000021f8c8e4/0x416193e012b04854 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762841.814427:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82086300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015045248:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6645us (101200us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762841.814444:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 832 00000100:00100000:0.0:1789762841.814452:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365111577068 time 365111660694 state 0x2 ref 0 00000100:00100000:0.0:1789762841.816242:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762841.816283:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015045376 00000100:00100000:0.0:1789762841.816290:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015045376 00000100:00100000:0.0:1789762841.816327:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365111577068 time 365111660694 state 0x0 ref 1 00000100:00100000:0.0:1789762841.816338:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365211660694 time 365111660694 state 0x1 ref 1 00000100:00100000:1.0:1789762841.907856:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365211660694 time 365211742018 state 0x0 ref 1 00000100:00100000:1.0:1789762841.907887:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82084a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015045376:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762841.907941:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b0485b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.907963:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762841.907968:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762841.907976:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b0485b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762841.907999:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b0485b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.911530:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b0485b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.911549:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b0485b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.911557:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b0485b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.911564:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762841.911568:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b0485b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.911577:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b0485b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.911598:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b0485b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762841.911605:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b0485b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762841.911674:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82084a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015045376:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3795us (95430us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762841.911683:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 833 00000100:00100000:3.0:1789762841.911694:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365211660694 time 365211742018 state 0x2 ref 0 00000100:00100000:0.0:1789762841.913741:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762841.913790:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015045504 00000100:00100000:1.0:1789762841.913803:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015045504 00000100:00100000:1.0:1789762841.913861:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365211660694 time 365211742018 state 0x0 ref 1 00000100:00100000:1.0:1789762841.913878:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365311742018 time 365211742018 state 0x1 ref 1 00000100:00100000:1.0:1789762842.007940:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365311742018 time 365311828738 state 0x0 ref 1 00000100:00100000:1.0:1789762842.007991:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015045504:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:0.0:1789762842.008031:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762842.008043:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04862 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.008075:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762842.008080:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762842.008111:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04862 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.008126:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04862 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.011500:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04862 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.011527:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04862 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.011536:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04862 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.011541:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762842.011545:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04862 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.011554:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04862 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.011573:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04862 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.011579:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04862 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762842.011650:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015045504:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3690us (97907us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762842.011659:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 834 00000100:00100000:1.0:1789762842.011670:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365311742018 time 365311828738 state 0x2 ref 0 00000100:00100000:0.0:1789762842.013496:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762842.013557:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015045632 00000100:00100000:2.0:1789762842.013567:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015045632 00000100:00100000:2.0:1789762842.013618:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365311742018 time 365311828738 state 0x0 ref 1 00000100:00100000:2.0:1789762842.013634:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365411828738 time 365311828738 state 0x1 ref 1 00000100:00100000:0.0:1789762842.108026:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365411828738 time 365411915688 state 0x0 ref 1 00000100:00100000:0.0:1789762842.108066:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015045632:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762842.108147:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d524ca57/0x416193e012b04869 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.108166:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762842.108170:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762842.108176:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d524ca57/0x416193e012b04869 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.108189:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d524ca57/0x416193e012b04869 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.111352:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d524ca57/0x416193e012b04869 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.111363:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d524ca57/0x416193e012b04869 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.111370:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d524ca57/0x416193e012b04869 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.111377:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c0e200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762842.111382:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d524ca57/0x416193e012b04869 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.111389:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d524ca57/0x416193e012b04869 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.111406:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d524ca57/0x416193e012b04869 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.111412:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d524ca57/0x416193e012b04869 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762842.111473:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015045632:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3430us (97981us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762842.111481:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 835 00000100:00100000:0.0:1789762842.111496:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365411828738 time 365411915688 state 0x2 ref 0 00000100:00100000:0.0:1789762842.113317:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762842.113402:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015045760 00000100:00100000:1.0:1789762842.113416:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015045760 00000100:00100000:1.0:1789762842.113469:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365411828738 time 365411915688 state 0x0 ref 1 00000100:00100000:1.0:1789762842.113481:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365511915688 time 365411915688 state 0x1 ref 1 00000100:00100000:3.0:1789762842.208149:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365511915688 time 365512043696 state 0x0 ref 1 00000100:00100000:3.0:1789762842.208183:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82084380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015045760:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762842.208224:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b04870 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.208238:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762842.208241:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762842.208246:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b04870 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.208257:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b04870 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.211474:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b04870 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.211483:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b04870 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.211489:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b04870 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.211495:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81682400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762842.211500:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b04870 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.211506:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b04870 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.211519:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b04870 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.211525:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df50b444/0x416193e012b04870 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762842.211575:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82084380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015045760:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3408us (98259us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762842.211582:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 836 00000100:00100000:3.0:1789762842.211588:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365511915688 time 365512043696 state 0x2 ref 0 00000100:00100000:2.0:1789762842.213646:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762842.213697:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015045888 00000100:00100000:1.0:1789762842.213705:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015045888 00000100:00100000:1.0:1789762842.213742:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365511915688 time 365512043696 state 0x0 ref 1 00000100:00100000:1.0:1789762842.213753:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365612043696 time 365512043696 state 0x1 ref 1 00000001:00080000:1.0:1789762842.240356:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967553 is committed 00000001:00080000:1.0:1789762842.240372:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967554 is committed 00000100:00100000:1.0:1789762842.308240:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365612043696 time 365612131691 state 0x0 ref 1 00000100:00100000:1.0:1789762842.308270:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015045888:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762842.308315:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04877 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.308332:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762842.308337:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762842.308344:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04877 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.308359:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04877 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.311748:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04877 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.311762:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04877 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.311772:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04877 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.311783:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762842.311789:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04877 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.311799:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04877 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.311818:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04877 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.311827:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04877 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762842.311909:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015045888:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3649us (98262us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762842.311919:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 837 00000100:00100000:1.0:1789762842.311930:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365612043696 time 365612131691 state 0x2 ref 0 00000100:00100000:2.0:1789762842.313877:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762842.313924:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015046016 00000100:00100000:1.0:1789762842.313933:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015046016 00000100:00100000:1.0:1789762842.313974:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365612043696 time 365612131691 state 0x0 ref 1 00000100:00100000:1.0:1789762842.313989:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365712131691 time 365612131691 state 0x1 ref 1 00000100:00100000:0.0:1789762842.408330:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365712131691 time 365712226288 state 0x0 ref 1 00000100:00100000:0.0:1789762842.408353:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015046016:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762842.408390:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b023c970/0x416193e012b0487e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.408403:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762842.408407:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762842.408411:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b023c970/0x416193e012b0487e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.408423:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b023c970/0x416193e012b0487e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.411792:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b023c970/0x416193e012b0487e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.411803:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b023c970/0x416193e012b0487e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.411811:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b023c970/0x416193e012b0487e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.411830:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc5a5a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762842.411834:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b023c970/0x416193e012b0487e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.411840:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b023c970/0x416193e012b0487e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.411856:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b023c970/0x416193e012b0487e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.411861:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b023c970/0x416193e012b0487e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762842.411912:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015046016:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3566us (98038us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762842.411920:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 838 00000100:00100000:0.0:1789762842.411928:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365712131691 time 365712226288 state 0x2 ref 0 00000100:00100000:0.0:1789762842.413912:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762842.413965:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015046144 00000100:00100000:1.0:1789762842.413975:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015046144 00000100:00100000:1.0:1789762842.414023:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365712131691 time 365712226288 state 0x0 ref 1 00000100:00100000:1.0:1789762842.414038:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365812226288 time 365712226288 state 0x1 ref 1 00000100:00100000:3.0:1789762842.508443:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365812226288 time 365812333083 state 0x0 ref 1 00000100:00100000:3.0:1789762842.508469:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82087b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015046144:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762842.508511:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b04885 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.508524:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762842.508528:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762842.508533:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b04885 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.508544:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b04885 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.511508:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b04885 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.511522:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b04885 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.511533:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b04885 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.511541:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c0ec00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762842.511547:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b04885 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.511558:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b04885 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.511573:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b04885 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.511579:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b04885 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762842.511632:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82087b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015046144:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3173us (97720us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762842.511639:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 839 00000100:00100000:3.0:1789762842.511646:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365812226288 time 365812333083 state 0x2 ref 0 00000100:00100000:3.0:1789762842.513514:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762842.513564:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015046272 00000100:00100000:1.0:1789762842.513573:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015046272 00000100:00100000:1.0:1789762842.513618:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365812226288 time 365812333083 state 0x0 ref 1 00000100:00100000:1.0:1789762842.513633:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365912333083 time 365812333083 state 0x1 ref 1 00010000:00010000:0.0:1789762842.551215:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762842.608514:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365912333083 time 365912405199 state 0x0 ref 1 00000100:00100000:1.0:1789762842.608546:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015046272:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762842.608587:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0488c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.608609:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762842.608614:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762842.608621:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0488c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.608636:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0488c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.611394:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0488c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.611414:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0488c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.611425:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0488c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.611433:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762842.611439:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0488c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.611448:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0488c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.611468:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0488c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.611484:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0488c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762842.611549:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015046272:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3014us (98033us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762842.611561:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 840 00000100:00100000:1.0:1789762842.611572:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365912333083 time 365912405199 state 0x2 ref 0 00000100:00100000:3.0:1789762842.613531:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762842.613574:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015046400 00000100:00100000:1.0:1789762842.613582:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015046400 00000100:00100000:1.0:1789762842.613622:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 365912333083 time 365912405199 state 0x0 ref 1 00000100:00100000:1.0:1789762842.613638:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366012405199 time 365912405199 state 0x1 ref 1 00000100:00100000:0.0:1789762842.708587:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366012405199 time 366012483066 state 0x0 ref 1 00000100:00100000:0.0:1789762842.708611:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015046400:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762842.708648:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04893 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.708674:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762842.708677:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762842.708682:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04893 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762842.708695:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04893 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.712111:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04893 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.712122:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04893 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.712129:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04893 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.712137:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762842.712141:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04893 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.712152:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04893 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.712170:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04893 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.712185:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04893 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762842.712237:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015046400:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3634us (98715us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762842.712245:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 841 00000100:00100000:3.0:1789762842.712253:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366012405199 time 366012483066 state 0x2 ref 0 00000100:00100000:3.0:1789762842.714202:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762842.714254:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015046528 00000100:00100000:1.0:1789762842.714264:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015046528 00000100:00100000:1.0:1789762842.714306:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366012405199 time 366012483066 state 0x0 ref 1 00000100:00100000:1.0:1789762842.714322:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366112483066 time 366012483066 state 0x1 ref 1 00000100:00100000:3.0:1789762842.808671:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366112483066 time 366112566843 state 0x0 ref 1 00000100:00100000:3.0:1789762842.808698:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015046528:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762842.808743:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b0489a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.808760:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762842.808764:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762842.808769:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b0489a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762842.808780:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b0489a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.811651:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b0489a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.811663:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b0489a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.811671:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b0489a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.811677:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c0ec00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762842.811681:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b0489a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.811688:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b0489a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.811703:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b0489a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.811709:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d4699326/0x416193e012b0489a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762842.811758:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015046528:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3071us (97555us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762842.811765:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 842 00000100:00100000:1.0:1789762842.811774:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366112483066 time 366112566843 state 0x2 ref 0 00000100:00100000:2.0:1789762842.813666:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762842.813704:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015046656 00000100:00100000:1.0:1789762842.813712:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015046656 00000100:00100000:1.0:1789762842.813748:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366112483066 time 366112566843 state 0x0 ref 1 00000100:00100000:1.0:1789762842.813762:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366212566843 time 366112566843 state 0x1 ref 1 00000100:00100000:1.0:1789762842.908726:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366212566843 time 366212621290 state 0x0 ref 1 00000100:00100000:1.0:1789762842.908750:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015046656:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762842.908787:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048a1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.908801:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762842.908805:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762842.908810:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048a1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.908821:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048a1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.911867:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048a1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.911878:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048a1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.911885:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048a1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.911893:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762842.911897:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048a1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.911903:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048a1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.911916:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048a1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762842.911921:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048a1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762842.911973:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015046656:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3232us (98308us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762842.911980:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 843 00000100:00100000:1.0:1789762842.911989:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366212566843 time 366212621290 state 0x2 ref 0 00000100:00100000:0.0:1789762842.914061:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762842.914115:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015046784 00000100:00100000:1.0:1789762842.914126:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015046784 00000100:00100000:1.0:1789762842.914169:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366212566843 time 366212621290 state 0x0 ref 1 00000100:00100000:1.0:1789762842.914181:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366312621290 time 366212621290 state 0x1 ref 1 00000100:00100000:3.0:1789762843.008803:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366312621290 time 366312696003 state 0x0 ref 1 00000100:00100000:3.0:1789762843.008836:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82084e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015046784:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762843.008878:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b048a8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762843.008891:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762843.008895:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762843.008900:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b048a8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762843.008910:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b048a8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.012703:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b048a8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.012723:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b048a8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.012733:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b048a8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.012742:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfcc00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762843.012747:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b048a8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.012756:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b048a8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.012775:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b048a8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.012791:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b048a8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762843.012866:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82084e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015046784:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4039us (98802us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762843.012878:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 844 00000100:00100000:0.0:1789762843.012892:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366312621290 time 366312696003 state 0x2 ref 0 00000100:00100000:0.0:1789762843.014876:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762843.014956:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015046912 00000100:00100000:1.0:1789762843.014965:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015046912 00000100:00100000:1.0:1789762843.015005:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366312621290 time 366312696003 state 0x0 ref 1 00000100:00100000:1.0:1789762843.015017:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366412696003 time 366312696003 state 0x1 ref 1 00000100:00100000:1.0:1789762843.108885:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366412696003 time 366412775653 state 0x0 ref 1 00000100:00100000:1.0:1789762843.108908:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82086a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015046912:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762843.108946:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048af lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.108961:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762843.108964:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762843.108969:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048af lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.108981:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048af lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.112270:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048af lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.112282:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048af lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.112289:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048af lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.112296:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762843.112299:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048af lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.112305:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048af lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.112319:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048af lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.112323:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048af lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762843.112374:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82086a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015046912:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3473us (97498us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762843.112382:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 845 00000100:00100000:1.0:1789762843.112390:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366412696003 time 366412775653 state 0x2 ref 0 00000100:00100000:0.0:1789762843.114270:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762843.114304:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015047040 00000100:00100000:1.0:1789762843.114312:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015047040 00000100:00100000:1.0:1789762843.114348:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366412696003 time 366412775653 state 0x0 ref 1 00000100:00100000:1.0:1789762843.114360:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366512775653 time 366412775653 state 0x1 ref 1 00000100:00100000:1.0:1789762843.208943:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366512775653 time 366512840186 state 0x0 ref 1 00000100:00100000:1.0:1789762843.208968:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82087800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015047040:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762843.209005:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048b6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.209019:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762843.209022:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762843.209027:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048b6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.209052:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048b6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.212135:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048b6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.212146:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048b6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.212154:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048b6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.212160:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762843.212163:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048b6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.212174:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048b6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.212190:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048b6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.212195:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048b6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762843.212246:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82087800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015047040:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3286us (97976us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762843.212253:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 846 00000100:00100000:1.0:1789762843.212260:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366512775653 time 366512840186 state 0x2 ref 0 00000100:00100000:3.0:1789762843.214279:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762843.214313:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015047168 00000100:00100000:1.0:1789762843.214323:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015047168 00000100:00100000:1.0:1789762843.214367:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366512775653 time 366512840186 state 0x0 ref 1 00000100:00100000:1.0:1789762843.214379:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366612840186 time 366512840186 state 0x1 ref 1 00000100:00100000:0.0:1789762843.309057:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366612840186 time 366612941176 state 0x0 ref 1 00000100:00100000:0.0:1789762843.309107:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015047168:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762843.309146:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b048bd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.309161:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762843.309164:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762843.309169:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b048bd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.309181:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b048bd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.312211:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b048bd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.312223:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b048bd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.312229:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b048bd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.312234:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbfc00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762843.312238:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b048bd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.312247:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b048bd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.312267:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b048bd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.312276:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000437d5020/0x416193e012b048bd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762843.312340:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015047168:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3265us (98060us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762843.312349:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 847 00000100:00100000:0.0:1789762843.312368:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366612840186 time 366612941176 state 0x2 ref 0 00000100:00100000:3.0:1789762843.314297:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762843.314330:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015047296 00000100:00100000:0.0:1789762843.314339:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015047296 00000100:00100000:0.0:1789762843.314383:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366612840186 time 366612941176 state 0x0 ref 1 00000100:00100000:0.0:1789762843.314395:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366712941176 time 366612941176 state 0x1 ref 1 00000100:00100000:1.0:1789762843.409212:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366712941176 time 366713103771 state 0x0 ref 1 00000100:00100000:1.0:1789762843.409243:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015047296:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762843.409284:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048c4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.409302:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762843.409307:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762843.409314:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048c4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.409328:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048c4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.412382:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048c4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.412396:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048c4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.412406:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048c4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.412416:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762843.412422:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048c4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.412432:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048c4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.412453:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048c4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.412469:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048c4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762843.412532:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015047296:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3299us (98235us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762843.412544:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 848 00000100:00100000:1.0:1789762843.412554:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366712941176 time 366713103771 state 0x2 ref 0 00000100:00100000:3.0:1789762843.414420:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762843.414462:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015047424 00000100:00100000:1.0:1789762843.414471:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015047424 00000100:00100000:1.0:1789762843.414513:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366712941176 time 366713103771 state 0x0 ref 1 00000100:00100000:1.0:1789762843.414527:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366813103771 time 366713103771 state 0x1 ref 1 00000100:00100000:1.0:1789762843.509280:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366813103771 time 366813174800 state 0x0 ref 1 00000100:00100000:1.0:1789762843.509305:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015047424:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762843.509342:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048cb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.509356:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762843.509359:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762843.509364:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048cb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.509380:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048cb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.512974:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048cb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.512987:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048cb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.512995:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048cb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.513000:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762843.513004:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048cb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.513011:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048cb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.513025:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048cb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.513030:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048cb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762843.513133:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015047424:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3836us (98712us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762843.513142:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 849 00000100:00100000:1.0:1789762843.513151:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366813103771 time 366813174800 state 0x2 ref 0 00000100:00100000:2.0:1789762843.515345:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762843.515418:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015047552 00000100:00100000:1.0:1789762843.515427:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015047552 00000100:00100000:1.0:1789762843.515465:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366813103771 time 366813174800 state 0x0 ref 1 00000100:00100000:1.0:1789762843.515476:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366913174800 time 366813174800 state 0x1 ref 1 00010000:00010000:1.0:1789762843.575228:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762843.609384:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366913174800 time 366913275408 state 0x0 ref 1 00000100:00100000:0.0:1789762843.609424:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015047552:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762843.609468:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048d2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.609485:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762843.609490:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762843.609496:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048d2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.609510:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048d2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.612557:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048d2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.612569:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048d2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.612586:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048d2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.612592:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762843.612595:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048d2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.612609:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048d2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.612625:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048d2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.612631:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048d2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762843.612679:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015047552:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3268us (97334us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762843.612688:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 850 00000100:00100000:0.0:1789762843.612696:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366913174800 time 366913275408 state 0x2 ref 0 00000100:00100000:0.0:1789762843.614551:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762843.614601:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015047680 00000100:00100000:2.0:1789762843.614609:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015047680 00000100:00100000:2.0:1789762843.614645:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 366913174800 time 366913275408 state 0x0 ref 1 00000100:00100000:2.0:1789762843.614656:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367013275408 time 366913275408 state 0x1 ref 1 00000100:00100000:1.0:1789762843.709490:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367013275408 time 367013386311 state 0x0 ref 1 00000100:00100000:1.0:1789762843.709523:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015047680:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762843.709567:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048d9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.709582:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762843.709585:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762843.709590:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048d9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.709602:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048d9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.712711:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048d9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.712722:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048d9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.712729:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048d9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.712734:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762843.712737:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048d9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.712743:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048d9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.712756:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048d9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.712761:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048d9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762843.712812:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015047680:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3301us (98260us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762843.712821:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 851 00000100:00100000:1.0:1789762843.712832:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367013275408 time 367013386311 state 0x2 ref 0 00000100:00100000:3.0:1789762843.714966:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762843.715007:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015047808 00000100:00100000:1.0:1789762843.715014:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015047808 00000100:00100000:1.0:1789762843.715063:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367013275408 time 367013386311 state 0x0 ref 1 00000100:00100000:1.0:1789762843.715073:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367113386311 time 367013386311 state 0x1 ref 1 00000100:00100000:0.0:1789762843.746134:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762843.746167:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762843.746175:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015048064 00000100:00100000:1.0:1789762843.746183:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015048064 00000100:00100000:0.0:1789762843.746187:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762843.746211:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015047936 00000100:00100000:3.0:1789762843.746217:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015047936 00000100:00100000:0.0:1789762843.746239:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762843.746241:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015048320 00000100:00100000:1.0:1789762843.746246:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015048320 00000100:00100000:2.0:1789762843.746267:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 278 00000100:00100000:3.0:1789762843.746268:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 316 00000100:00100000:0.0:1789762843.746273:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015048192 00000100:00100000:0.0:1789762843.746279:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015048192 00000100:00100000:2.0:1789762843.746287:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82087100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702015048064:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762843.746287:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85054850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015047936:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762843.746295:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 279 00000100:00100000:1.0:1789762843.746308:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82084000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015048320:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762843.746322:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 141 00000100:00100000:0.0:1789762843.746333:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82086d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015048192:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762843.746344:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85054850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015047936:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 65us (175us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762843.746354:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 316 00000100:00100000:2.0:1789762843.746355:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82087100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702015048064:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 72us (221us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762843.746362:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82084000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015048320:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 55us (171us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762843.746369:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 278 00000100:00100000:0.0:1789762843.746371:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82086d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015048192:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 40us (133us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762843.746378:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 279 00000100:00100000:0.0:1789762843.746382:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 141 00000100:00080000:2.0:1789762843.746399:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762843.746405:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762843.767290:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022174336:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762843.767359:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762843.767396:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022174336 00000100:00100000:3.0:1789762843.767405:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022174336 00000100:00080000:2.0:1789762843.767458:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762843.767475:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:3.0:1789762843.767489:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022174336:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762843.767523:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:3.0:1789762843.767568:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022174336:12345-0@lo:13:osp-pre-0-0.0 Request processed in 84us (208us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762843.767574:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00000100:00100000:0.0:1789762843.767612:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022174336:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762843.809564:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367113386311 time 367113456853 state 0x0 ref 1 00000100:00100000:1.0:1789762843.809593:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015047808:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762843.809635:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048e0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.809650:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762843.809654:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762843.809659:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048e0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.809671:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048e0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.812812:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048e0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.812826:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048e0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.812836:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048e0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.812846:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762843.812850:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048e0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.812859:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048e0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.812879:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048e0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762843.812887:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048e0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762843.812955:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015047808:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3372us (97988us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762843.812966:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 852 00000100:00100000:1.0:1789762843.812977:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367113386311 time 367113456853 state 0x2 ref 0 00000100:00100000:0.0:1789762843.815057:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762843.815112:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015048448 00000100:00100000:1.0:1789762843.815121:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015048448 00000100:00100000:1.0:1789762843.815158:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367113386311 time 367113456853 state 0x0 ref 1 00000100:00100000:1.0:1789762843.815169:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367213456853 time 367113456853 state 0x1 ref 1 00000100:00100000:0.0:1789762843.909658:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367213456853 time 367213553898 state 0x0 ref 1 00000100:00100000:0.0:1789762843.909686:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015048448:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762843.909730:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048e7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.909744:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762843.909747:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762843.909752:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048e7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.909764:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048e7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.912500:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048e7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.912512:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048e7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.912521:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048e7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.912529:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762843.912533:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048e7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.912542:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048e7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.912560:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048e7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762843.912569:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009379584a/0x416193e012b048e7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762843.912622:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015048448:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2948us (97566us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762843.912630:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 853 00000100:00100000:0.0:1789762843.912644:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367213456853 time 367213553898 state 0x2 ref 0 00000100:00100000:0.0:1789762843.914809:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762843.914850:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015048576 00000100:00100000:2.0:1789762843.914858:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015048576 00000100:00100000:2.0:1789762843.914895:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367213456853 time 367213553898 state 0x0 ref 1 00000100:00100000:2.0:1789762843.914907:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367313553898 time 367213553898 state 0x1 ref 1 00000100:00100000:1.0:1789762844.009761:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367313553898 time 367313656530 state 0x0 ref 1 00000100:00100000:1.0:1789762844.009799:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84543800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015048576:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:2.0:1789762844.009831:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762844.009836:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048ee lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.009852:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.009856:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.009860:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048ee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.009872:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048ee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.012729:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048ee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.012740:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048ee lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.012748:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048ee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.012754:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762844.012759:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048ee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.012765:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048ee lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.012780:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048ee lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.012785:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048ee lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762844.012832:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015048576:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3057us (98025us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762844.012840:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 854 00000100:00100000:1.0:1789762844.012847:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367313553898 time 367313656530 state 0x2 ref 0 00000100:00100000:3.0:1789762844.014724:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762844.014756:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015048704 00000100:00100000:1.0:1789762844.014763:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015048704 00000100:00100000:1.0:1789762844.014797:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367313553898 time 367313656530 state 0x0 ref 1 00000100:00100000:1.0:1789762844.014807:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367413656530 time 367313656530 state 0x1 ref 1 00000100:00100000:1.0:1789762844.109831:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367413656530 time 367413727421 state 0x0 ref 1 00000100:00100000:1.0:1789762844.109853:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015048704:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762844.109889:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048f5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.109903:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.109906:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.109911:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048f5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.109922:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048f5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.113227:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048f5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.113240:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048f5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.113250:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048f5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.113259:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762844.113264:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048f5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.113274:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048f5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.113306:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048f5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.113317:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b048f5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762844.113371:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015048704:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3526us (98647us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762844.113377:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 855 00000100:00100000:1.0:1789762844.113384:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367413656530 time 367413727421 state 0x2 ref 0 00000100:00100000:3.0:1789762844.115333:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762844.115369:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015048832 00000100:00100000:1.0:1789762844.115378:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015048832 00000100:00100000:1.0:1789762844.115414:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367413656530 time 367413727421 state 0x0 ref 1 00000100:00100000:1.0:1789762844.115428:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367513727421 time 367413727421 state 0x1 ref 1 00000100:00100000:0.0:1789762844.209920:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367513727421 time 367513816453 state 0x0 ref 1 00000100:00100000:0.0:1789762844.209945:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015048832:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762844.209984:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b048fc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.209997:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762844.210000:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762844.210005:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b048fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.210016:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b048fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.213293:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b048fc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.213304:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b048fc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.213311:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b048fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.213334:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762844.213338:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b048fc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.213345:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b048fc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.213357:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b048fc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.213362:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b048fc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762844.213411:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015048832:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3476us (98079us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762844.213419:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 856 00000100:00100000:0.0:1789762844.213426:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367513727421 time 367513816453 state 0x2 ref 0 00000100:00100000:0.0:1789762844.215502:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762844.215539:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015048960 00000100:00100000:2.0:1789762844.215547:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015048960 00000100:00100000:2.0:1789762844.215589:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367513727421 time 367513816453 state 0x0 ref 1 00000100:00100000:2.0:1789762844.215600:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367613816453 time 367513816453 state 0x1 ref 1 00000100:00100000:2.0:1789762844.279240:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d84542680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022174464:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762844.279280:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb8ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022174592:0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.279316:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762844.279323:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb8fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022174848:0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.279335:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762844.279349:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb8ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022174720:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.279375:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762844.279398:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb8d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022174976:0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.279406:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022175104:0@lo:400:kworker.0 00000100:00100000:3.0:1789762844.279434:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762844.279452:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762844.279453:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022174848 00000100:00100000:1.0:1789762844.279461:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022174848 00000100:00100000:2.0:1789762844.279464:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022174592 00000100:00100000:2.0:1789762844.279471:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022174592 00000100:00100000:3.0:1789762844.279533:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022174464 00000100:00100000:3.0:1789762844.279541:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022174464 00000100:00100000:0.0:1789762844.279542:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022175104 00000100:00100000:2.0:1789762844.279544:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:1.0:1789762844.279554:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022174720 00000100:00100000:2.0:1789762844.279557:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85056450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022174592:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.279562:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022175104 00000100:00100000:1.0:1789762844.279563:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022174720 00000100:00100000:3.0:1789762844.279602:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 118 00000100:00100000:1.0:1789762844.279619:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:0.0:1789762844.279630:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:3.0:1789762844.279631:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022174464:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762844.279633:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022174848:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.279644:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85056450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022174592:12345-0@lo:400:kworker.0 Request processed in 86us (303us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762844.279654:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022175104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762844.279669:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00002000:00000024:3.0:1789762844.279705:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762844.279725:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022174848:12345-0@lo:400:kworker.0 Request processed in 93us (349us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762844.279730:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022175104:12345-0@lo:400:kworker.0 Request processed in 81us (277us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762844.279736:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:2.0:1789762844.279736:0:3271:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967416 gen 2 00000100:00100000:0.0:1789762844.279741:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000004:00080000:2.0:1789762844.279745:0:3271:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981d8257d500, transno 4294967416 00000100:00100000:2.0:1789762844.279771:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb8c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022175104:0@lo:400:kworker.0 00000100:00100000:3.0:1789762844.279777:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022174464:12345-0@lo:13:osp-pre-1-0.0 Request processed in 159us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762844.279778:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 281 00000100:00080000:0.0:1789762844.279793:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762844.279801:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762844.279805:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 118 00000100:00100000:2.0:1789762844.279806:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb8ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022174592:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.279810:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022174720:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762844.279864:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb8fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022174848:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.279885:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022174720:12345-0@lo:400:kworker.0 Request processed in 84us (449us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762844.279886:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762844.279900:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 281 00000100:00100000:0.0:1789762844.279930:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb8ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022174720:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.279931:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022174976 00000100:00100000:1.0:1789762844.279937:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022174976 00000100:00100000:3.0:1789762844.279965:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d84542680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022174464:0@lo:13:osp-pre-1-0.0 00000100:00080000:0.0:1789762844.279972:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762844.279989:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1789762844.279997:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022174976:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.280057:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022174976:12345-0@lo:400:kworker.0 Request processed in 62us (178us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762844.280063:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1789762844.280126:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb8d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022174976:0@lo:400:kworker.0 00000100:00100000:1.0:1789762844.310059:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367613816453 time 367613953103 state 0x0 ref 1 00000100:00100000:1.0:1789762844.310114:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015048960:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762844.310157:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04903 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.310171:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.310175:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.310180:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04903 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.310191:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04903 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.313425:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04903 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.313436:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04903 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.313446:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04903 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.313455:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762844.313460:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04903 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.313471:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04903 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.313487:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04903 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.313495:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04903 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762844.313553:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015048960:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3476us (98050us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762844.313563:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 857 00000100:00100000:1.0:1789762844.313570:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367613816453 time 367613953103 state 0x2 ref 0 00000100:00100000:0.0:1789762844.315647:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762844.315714:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015049088 00000100:00100000:1.0:1789762844.315723:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015049088 00000100:00100000:1.0:1789762844.315759:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367613816453 time 367613953103 state 0x0 ref 1 00000100:00100000:1.0:1789762844.315771:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367713953103 time 367613953103 state 0x1 ref 1 00000100:00100000:1.0:1789762844.410155:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367713953103 time 367714046606 state 0x0 ref 1 00000100:00100000:1.0:1789762844.410186:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015049088:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762844.410228:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0490a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.410247:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.410252:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.410259:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0490a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.410274:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0490a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.413294:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0490a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.413309:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0490a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.413319:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0490a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.413328:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762844.413333:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0490a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.413343:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0490a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.413362:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0490a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.413373:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0490a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762844.413424:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015049088:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3251us (97778us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762844.413431:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 858 00000100:00100000:1.0:1789762844.413439:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367713953103 time 367714046606 state 0x2 ref 0 00000100:00100000:3.0:1789762844.415333:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762844.415373:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015049216 00000100:00100000:1.0:1789762844.415381:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015049216 00000100:00100000:1.0:1789762844.415418:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367713953103 time 367714046606 state 0x0 ref 1 00000100:00100000:1.0:1789762844.415429:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367814046606 time 367714046606 state 0x1 ref 1 00000100:00100000:0.0:1789762844.510280:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367814046606 time 367814171652 state 0x0 ref 1 00000100:00100000:0.0:1789762844.510312:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015049216:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762844.510354:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04911 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.510370:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762844.510375:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762844.510382:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04911 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.510397:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04911 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.513500:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04911 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.513513:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04911 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.513523:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04911 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.513531:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762844.513536:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04911 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.513546:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04911 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.513563:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04911 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.513572:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04911 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762844.513635:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015049216:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3334us (98306us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762844.513646:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 859 00000100:00100000:0.0:1789762844.513664:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367814046606 time 367814171652 state 0x2 ref 0 00000100:00100000:2.0:1789762844.515545:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762844.515589:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015049344 00000100:00100000:0.0:1789762844.515598:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015049344 00000100:00100000:0.0:1789762844.515640:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367814046606 time 367814171652 state 0x0 ref 1 00000100:00100000:0.0:1789762844.515655:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367914171652 time 367814171652 state 0x1 ref 1 00010000:00010000:0.0:1789762844.599218:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762844.610395:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367914171652 time 367914288203 state 0x0 ref 1 00000100:00100000:1.0:1789762844.610424:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015049344:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762844.610475:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04918 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.610494:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.610499:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.610506:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04918 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.610527:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04918 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.614291:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04918 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.614304:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04918 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.614315:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04918 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.614325:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762844.614331:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04918 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.614349:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04918 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.614368:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04918 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.614377:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04918 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762844.614438:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015049344:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4025us (98894us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762844.614445:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 860 00000100:00100000:1.0:1789762844.614453:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367914171652 time 367914288203 state 0x2 ref 0 00000100:00100000:2.0:1789762844.616519:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762844.616551:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015049472 00000100:00100000:1.0:1789762844.616558:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015049472 00000100:00100000:1.0:1789762844.616590:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 367914171652 time 367914288203 state 0x0 ref 1 00000100:00100000:1.0:1789762844.616600:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368014288203 time 367914288203 state 0x1 ref 1 00000100:00100000:1.0:1789762844.710493:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368014288203 time 368014381255 state 0x0 ref 1 00000100:00100000:1.0:1789762844.710523:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015049472:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762844.710570:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0491f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.710589:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762844.710594:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.710600:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0491f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.710618:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0491f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.714272:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0491f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.714290:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0491f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.714300:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0491f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.714308:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762844.714313:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0491f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.714322:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0491f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.714342:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0491f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.714350:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0491f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762844.714423:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015049472:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3912us (97903us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762844.714432:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 861 00000100:00100000:1.0:1789762844.714442:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368014288203 time 368014381255 state 0x2 ref 0 00000100:00100000:2.0:1789762844.716466:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762844.716507:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015049600 00000100:00100000:1.0:1789762844.716515:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015049600 00000100:00100000:1.0:1789762844.716552:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368014288203 time 368014381255 state 0x0 ref 1 00000100:00100000:1.0:1789762844.716562:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368114381255 time 368014381255 state 0x1 ref 1 00000100:00100000:0.0:1789762844.810654:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368114381255 time 368114544132 state 0x0 ref 1 00000100:00100000:0.0:1789762844.810686:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015049600:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762844.810728:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04926 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.810745:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762844.810750:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762844.810756:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04926 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.810771:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04926 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.814471:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04926 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.814503:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04926 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.814512:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04926 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.814522:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762844.814527:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04926 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.814538:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04926 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.814558:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04926 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762844.814567:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04926 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762844.814628:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015049600:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3953us (98171us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762844.814639:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 862 00000100:00100000:0.0:1789762844.814650:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368114381255 time 368114544132 state 0x2 ref 0 00000100:00100000:3.0:1789762844.816858:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762844.816907:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015049728 00000100:00100000:0.0:1789762844.816923:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015049728 00000100:00100000:0.0:1789762844.816965:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368114381255 time 368114544132 state 0x0 ref 1 00000100:00100000:0.0:1789762844.816980:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368214544132 time 368114544132 state 0x1 ref 1 00000100:00100000:1.0:1789762844.910776:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368214544132 time 368214666825 state 0x0 ref 1 00000100:00100000:1.0:1789762844.910806:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015049728:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762844.910847:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0492d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.910866:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.910871:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762844.910878:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0492d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.910893:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0492d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.914076:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0492d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.914123:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0492d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.914133:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0492d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.914139:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762844.914144:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0492d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.914153:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0492d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.914181:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0492d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762844.914192:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0492d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762844.914277:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015049728:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3480us (97425us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762844.914286:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 863 00000100:00100000:1.0:1789762844.914298:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368214544132 time 368214666825 state 0x2 ref 0 00000100:00100000:3.0:1789762844.916277:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762844.916312:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015049856 00000100:00100000:1.0:1789762844.916321:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015049856 00000100:00100000:1.0:1789762844.916368:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368214544132 time 368214666825 state 0x0 ref 1 00000100:00100000:1.0:1789762844.916392:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368314666825 time 368214666825 state 0x1 ref 1 00000100:00100000:1.0:1789762845.010864:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368314666825 time 368314752063 state 0x0 ref 1 00000100:00100000:1.0:1789762845.010895:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015049856:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762845.010946:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04934 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.010963:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.010967:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.010973:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04934 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.010987:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04934 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.014341:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04934 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.014358:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04934 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.014370:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04934 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.014383:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762845.014388:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04934 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.014397:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04934 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.014416:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04934 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.014421:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04934 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762845.014489:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015049856:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3607us (98212us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762845.014500:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 864 00000100:00100000:1.0:1789762845.014511:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368314666825 time 368314752063 state 0x2 ref 0 00000100:00100000:2.0:1789762845.016400:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762845.016435:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015049984 00000100:00100000:1.0:1789762845.016444:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015049984 00000100:00100000:1.0:1789762845.016485:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368314666825 time 368314752063 state 0x0 ref 1 00000100:00100000:1.0:1789762845.016498:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368414752063 time 368314752063 state 0x1 ref 1 00000100:00100000:0.0:1789762845.110954:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368414752063 time 368414849655 state 0x0 ref 1 00000100:00100000:0.0:1789762845.110985:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015049984:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762845.111020:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b0493b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.111170:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762845.111174:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762845.111179:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b0493b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.111191:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b0493b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.114400:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b0493b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.114411:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b0493b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.114418:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b0493b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.114437:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762845.114440:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b0493b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.114447:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b0493b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.114460:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b0493b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.114465:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b0493b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762845.114514:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015049984:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3537us (98116us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762845.114521:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 865 00000100:00100000:0.0:1789762845.114530:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368414752063 time 368414849655 state 0x2 ref 0 00000100:00100000:0.0:1789762845.116454:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762845.116516:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015050112 00000100:00100000:2.0:1789762845.116525:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015050112 00000100:00100000:2.0:1789762845.116563:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368414752063 time 368414849655 state 0x0 ref 1 00000100:00100000:2.0:1789762845.116574:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368514849655 time 368414849655 state 0x1 ref 1 00000100:00100000:1.0:1789762845.211081:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368514849655 time 368514965193 state 0x0 ref 1 00000100:00100000:1.0:1789762845.211140:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015050112:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762845.211199:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04942 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.211220:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.211225:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.211231:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04942 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.211245:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04942 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.214274:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04942 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.214292:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04942 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.214303:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04942 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.214310:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762845.214315:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04942 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.214324:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04942 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.214344:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04942 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.214350:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04942 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762845.214412:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015050112:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3286us (97959us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762845.214419:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 866 00000100:00100000:1.0:1789762845.214429:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368514849655 time 368514965193 state 0x2 ref 0 00000100:00100000:3.0:1789762845.216299:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762845.216340:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015050240 00000100:00100000:1.0:1789762845.216352:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015050240 00000100:00100000:1.0:1789762845.216404:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368514849655 time 368514965193 state 0x0 ref 1 00000100:00100000:1.0:1789762845.216422:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368614965193 time 368514965193 state 0x1 ref 1 00000100:00100000:1.0:1789762845.311145:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368614965193 time 368615038598 state 0x0 ref 1 00000100:00100000:1.0:1789762845.311178:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015050240:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762845.311231:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04949 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.311249:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.311254:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.311259:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04949 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.311272:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04949 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.314578:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04949 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.314589:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04949 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.314596:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04949 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.314601:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762845.314606:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04949 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.314612:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04949 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.314627:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04949 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.314632:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04949 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762845.314697:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015050240:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3531us (98399us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762845.314705:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 867 00000100:00100000:1.0:1789762845.314714:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368614965193 time 368615038598 state 0x2 ref 0 00000100:00100000:3.0:1789762845.316804:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762845.316870:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015050368 00000100:00100000:1.0:1789762845.316880:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015050368 00000100:00100000:1.0:1789762845.316921:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368614965193 time 368615038598 state 0x0 ref 1 00000100:00100000:1.0:1789762845.316933:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368715038598 time 368615038598 state 0x1 ref 1 00000100:00100000:0.0:1789762845.411239:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368715038598 time 368715132712 state 0x0 ref 1 00000100:00100000:0.0:1789762845.411266:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015050368:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762845.411311:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04950 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.411343:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762845.411347:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762845.411352:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04950 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.411366:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04950 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762845.414384:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04950 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762845.414400:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04950 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762845.414410:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04950 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762845.414420:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762845.414427:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04950 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762845.414439:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04950 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762845.414462:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04950 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762845.414471:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04950 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762845.414537:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc868380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015050368:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3278us (97732us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762845.414550:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 868 00000100:00100000:3.0:1789762845.414563:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368715038598 time 368715132712 state 0x2 ref 0 00000100:00100000:3.0:1789762845.416414:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762845.416466:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015050496 00000100:00100000:2.0:1789762845.416482:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015050496 00000100:00100000:2.0:1789762845.416544:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368715038598 time 368715132712 state 0x0 ref 1 00000100:00100000:2.0:1789762845.416557:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368815132712 time 368715132712 state 0x1 ref 1 00000100:00100000:1.0:1789762845.511333:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368815132712 time 368815223542 state 0x0 ref 1 00000100:00100000:1.0:1789762845.511364:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015050496:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762845.511413:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04957 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.511431:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.511436:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.511445:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04957 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.511462:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04957 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.514722:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04957 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.514733:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04957 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.514741:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04957 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.514746:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762845.514751:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04957 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.514757:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04957 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.514773:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04957 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.514778:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04957 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762845.514843:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc868a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015050496:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3492us (98430us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762845.514852:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 869 00000100:00100000:1.0:1789762845.514862:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368815132712 time 368815223542 state 0x2 ref 0 00000100:00100000:3.0:1789762845.516782:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762845.516849:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015050624 00000100:00100000:1.0:1789762845.516860:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015050624 00000100:00100000:1.0:1789762845.516904:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368815132712 time 368815223542 state 0x0 ref 1 00000100:00100000:1.0:1789762845.516917:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368915223542 time 368815223542 state 0x1 ref 1 00000100:00100000:1.0:1789762845.611395:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368915223542 time 368915286112 state 0x0 ref 1 00000100:00100000:1.0:1789762845.611424:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015050624:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762845.611477:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0495e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.611495:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.611501:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.611509:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0495e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.611526:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0495e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.614868:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0495e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.614885:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0495e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.614894:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0495e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.614900:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762845.614904:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0495e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.614911:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0495e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.614932:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0495e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.614938:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0495e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762845.615016:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015050624:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3597us (98229us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762845.615042:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 870 00000100:00100000:1.0:1789762845.615055:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368915223542 time 368915286112 state 0x2 ref 0 00000100:00100000:3.0:1789762845.617003:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762845.617054:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015050752 00000100:00100000:1.0:1789762845.617062:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015050752 00000100:00100000:1.0:1789762845.617119:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 368915223542 time 368915286112 state 0x0 ref 1 00000100:00100000:1.0:1789762845.617133:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369015286112 time 368915286112 state 0x1 ref 1 00010000:00010000:0.0:1789762845.623225:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762845.711489:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369015286112 time 369015379780 state 0x0 ref 1 00000100:00100000:3.0:1789762845.711521:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015050752:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762845.711569:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b04965 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762845.711588:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762845.711592:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762845.711598:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b04965 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762845.711620:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b04965 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.715529:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b04965 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.715559:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b04965 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.715585:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b04965 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.715594:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfde00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762845.715600:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b04965 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.715612:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b04965 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.715640:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b04965 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762845.715666:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b04965 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762845.715763:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015050752:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4250us (98755us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762845.715779:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 871 00000100:00100000:0.0:1789762845.715793:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369015286112 time 369015379780 state 0x2 ref 0 00000100:00100000:3.0:1789762845.717988:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762845.718059:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015050880 00000100:00100000:2.0:1789762845.718070:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015050880 00000100:00100000:2.0:1789762845.718175:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369015286112 time 369015379780 state 0x0 ref 1 00000100:00100000:2.0:1789762845.718193:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369115379780 time 369015379780 state 0x1 ref 1 00000100:00100000:1.0:1789762845.811612:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369115379780 time 369115506372 state 0x0 ref 1 00000100:00100000:1.0:1789762845.811639:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015050880:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762845.811677:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0496c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.811691:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.811695:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.811700:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0496c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.811711:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0496c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.814669:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0496c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.814678:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0496c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.814684:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0496c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.814693:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762845.814696:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0496c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.814703:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0496c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.814715:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0496c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.814721:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0496c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762845.814794:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc868700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015050880:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3165us (96808us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762845.814804:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 872 00000100:00100000:1.0:1789762845.814814:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369115379780 time 369115506372 state 0x2 ref 0 00000100:00100000:2.0:1789762845.816789:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762845.816827:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015051008 00000100:00100000:1.0:1789762845.816835:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015051008 00000100:00100000:1.0:1789762845.816868:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369115379780 time 369115506372 state 0x0 ref 1 00000100:00100000:1.0:1789762845.816878:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369215506372 time 369115506372 state 0x1 ref 1 00000100:00100000:1.0:1789762845.911695:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369215506372 time 369215586755 state 0x0 ref 1 00000100:00100000:1.0:1789762845.911725:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015051008:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762845.911762:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04973 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.911777:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.911781:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762845.911785:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04973 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.911796:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04973 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.915142:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04973 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.915153:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04973 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.915159:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04973 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.915164:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762845.915168:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04973 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.915178:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04973 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.915193:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04973 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762845.915199:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04973 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762845.915254:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015051008:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3540us (98466us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762845.915260:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 873 00000100:00100000:1.0:1789762845.915267:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369215506372 time 369215586755 state 0x2 ref 0 00000100:00100000:2.0:1789762845.917191:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762845.917231:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015051136 00000100:00100000:1.0:1789762845.917240:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015051136 00000100:00100000:1.0:1789762845.917275:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369215506372 time 369215586755 state 0x0 ref 1 00000100:00100000:1.0:1789762845.917285:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369315586755 time 369215586755 state 0x1 ref 1 00000100:00100000:0.0:1789762846.011790:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369315586755 time 369315680727 state 0x0 ref 1 00000100:00100000:0.0:1789762846.011849:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc4f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015051136:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:2.0:1789762846.011887:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762846.011901:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0497a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.011919:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762846.011923:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762846.011929:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0497a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.011943:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0497a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.015187:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0497a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.015223:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0497a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.015232:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0497a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.015238:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762846.015243:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0497a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.015250:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0497a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.015268:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0497a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.015274:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0497a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762846.015336:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc4f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015051136:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3527us (98145us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762846.015344:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 874 00000100:00100000:0.0:1789762846.015353:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369315586755 time 369315680727 state 0x2 ref 0 00000100:00100000:3.0:1789762846.017366:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762846.017430:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015051264 00000100:00100000:2.0:1789762846.017443:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015051264 00000100:00100000:2.0:1789762846.017500:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369315586755 time 369315680727 state 0x0 ref 1 00000100:00100000:2.0:1789762846.017513:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369415680727 time 369315680727 state 0x1 ref 1 00000100:00100000:1.0:1789762846.111905:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369415680727 time 369415799345 state 0x0 ref 1 00000100:00100000:1.0:1789762846.111931:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015051264:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762846.111967:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04981 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.111983:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762846.111986:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762846.111991:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04981 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.112003:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04981 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.115635:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04981 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.115654:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04981 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.115667:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04981 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.115675:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762846.115681:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04981 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.115692:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04981 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.115716:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04981 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.115730:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04981 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762846.115788:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015051264:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3867us (98422us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762846.115796:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 875 00000100:00100000:1.0:1789762846.115805:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369415680727 time 369415799345 state 0x2 ref 0 00000100:00100000:3.0:1789762846.118326:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762846.118363:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015051392 00000100:00100000:1.0:1789762846.118371:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015051392 00000100:00100000:1.0:1789762846.118412:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369415680727 time 369415799345 state 0x0 ref 1 00000100:00100000:1.0:1789762846.118424:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369515799345 time 369415799345 state 0x1 ref 1 00000100:00100000:1.0:1789762846.211993:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369515799345 time 369515885929 state 0x0 ref 1 00000100:00100000:1.0:1789762846.212038:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015051392:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762846.212082:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04988 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.212124:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762846.212129:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762846.212136:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04988 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.212158:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04988 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.215274:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04988 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.215287:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04988 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.215297:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04988 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.215306:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762846.215310:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04988 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.215317:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04988 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.215333:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04988 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.215339:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04988 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762846.215405:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015051392:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3395us (97086us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762846.215412:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 876 00000100:00100000:1.0:1789762846.215419:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369515799345 time 369515885929 state 0x2 ref 0 00000100:00100000:3.0:1789762846.217645:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762846.217687:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015051520 00000100:00100000:1.0:1789762846.217695:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015051520 00000100:00100000:1.0:1789762846.217729:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369515799345 time 369515885929 state 0x0 ref 1 00000100:00100000:1.0:1789762846.217739:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369615885929 time 369515885929 state 0x1 ref 1 00000100:00100000:0.0:1789762846.312167:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369615885929 time 369616057312 state 0x0 ref 1 00000100:00100000:0.0:1789762846.312212:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015051520:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762846.312248:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0498f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.312264:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762846.312268:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762846.312272:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0498f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.312283:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0498f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.315905:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0498f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.315917:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0498f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.315924:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0498f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.315929:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762846.315933:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0498f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.315938:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0498f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.315960:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0498f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.315966:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b0498f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762846.316032:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015051520:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3828us (98393us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762846.316043:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 877 00000100:00100000:0.0:1789762846.316052:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369615885929 time 369616057312 state 0x2 ref 0 00000100:00100000:2.0:1789762846.317892:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762846.317943:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015051648 00000100:00100000:0.0:1789762846.317952:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015051648 00000100:00100000:0.0:1789762846.317991:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369615885929 time 369616057312 state 0x0 ref 1 00000100:00100000:0.0:1789762846.318002:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369716057312 time 369616057312 state 0x1 ref 1 00000100:00100000:1.0:1789762846.412261:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369716057312 time 369716156357 state 0x0 ref 1 00000100:00100000:1.0:1789762846.412285:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015051648:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762846.412336:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04996 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.412354:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762846.412357:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762846.412362:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04996 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.412373:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04996 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.415760:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04996 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.415770:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04996 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.415777:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04996 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.415783:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762846.415786:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04996 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.415793:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04996 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.415807:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04996 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.415814:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04996 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762846.415881:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015051648:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3603us (97995us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762846.415888:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 878 00000100:00100000:1.0:1789762846.415897:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369716057312 time 369716156357 state 0x2 ref 0 00000100:00100000:3.0:1789762846.417618:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762846.417696:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015051776 00000100:00100000:1.0:1789762846.417705:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015051776 00000100:00100000:1.0:1789762846.417741:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369716057312 time 369716156357 state 0x0 ref 1 00000100:00100000:1.0:1789762846.417752:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369816156357 time 369716156357 state 0x1 ref 1 00000100:00100000:1.0:1789762846.512352:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369816156357 time 369816247979 state 0x0 ref 1 00000100:00100000:1.0:1789762846.512378:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015051776:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762846.512420:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0499d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.512440:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762846.512445:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762846.512452:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0499d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.512467:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0499d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.515957:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0499d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.515969:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0499d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.515976:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0499d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.515981:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762846.515984:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0499d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.515990:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0499d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.516003:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0499d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.516009:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b0499d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762846.516073:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc868e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015051776:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3705us (98464us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762846.516079:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 879 00000100:00100000:1.0:1789762846.516123:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369816156357 time 369816247979 state 0x2 ref 0 00000100:00100000:3.0:1789762846.517957:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762846.518000:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015051904 00000100:00100000:1.0:1789762846.518009:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015051904 00000100:00100000:1.0:1789762846.518061:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369816156357 time 369816247979 state 0x0 ref 1 00000100:00100000:1.0:1789762846.518075:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369916247979 time 369816247979 state 0x1 ref 1 00000100:00100000:0.0:1789762846.612453:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369916247979 time 369916344742 state 0x0 ref 1 00000100:00100000:0.0:1789762846.612485:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015051904:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762846.612524:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b049a4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.612542:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762846.612569:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762846.612576:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b049a4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.612588:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b049a4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.615901:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b049a4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.615912:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b049a4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.615919:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b049a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.615925:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762846.615928:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b049a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.615934:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b049a4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.615948:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b049a4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.615954:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000addc69eb/0x416193e012b049a4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762846.616033:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015051904:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3559us (98080us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762846.616045:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 880 00000100:00100000:0.0:1789762846.616052:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369916247979 time 369916344742 state 0x2 ref 0 00000100:00100000:2.0:1789762846.617925:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762846.617966:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015052032 00000100:00100000:0.0:1789762846.617974:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015052032 00000100:00100000:0.0:1789762846.618023:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 369916247979 time 369916344742 state 0x0 ref 1 00000100:00100000:0.0:1789762846.618035:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370016344742 time 369916344742 state 0x1 ref 1 00010000:00010000:0.0:1789762846.647174:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762846.712536:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370016344742 time 370016431745 state 0x0 ref 1 00000100:00100000:1.0:1789762846.712559:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015052032:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762846.712595:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ab lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.712608:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762846.712612:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762846.712616:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ab lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.712629:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ab lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.716054:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ab lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.716064:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ab lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.716070:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ab lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.716079:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762846.716082:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ab lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.716106:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ab lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.716118:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ab lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.716124:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ab lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762846.716178:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015052032:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3627us (98261us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762846.716184:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 881 00000100:00100000:1.0:1789762846.716193:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370016344742 time 370016431745 state 0x2 ref 0 00000100:00100000:3.0:1789762846.717937:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762846.718026:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015052160 00000100:00100000:1.0:1789762846.718036:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015052160 00000100:00100000:1.0:1789762846.718079:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370016344742 time 370016431745 state 0x0 ref 1 00000100:00100000:1.0:1789762846.718122:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370116431745 time 370016431745 state 0x1 ref 1 00000100:00100000:1.0:1789762846.812630:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370116431745 time 370116526045 state 0x0 ref 1 00000100:00100000:1.0:1789762846.812657:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015052160:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762846.812694:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049b2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.812708:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762846.812712:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762846.812717:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049b2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.812727:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049b2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.816319:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049b2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.816329:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049b2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.816337:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049b2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.816343:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762846.816346:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049b2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.816357:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049b2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.816371:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049b2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762846.816379:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049b2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762846.816448:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015052160:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3802us (98519us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762846.816455:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 882 00000100:00100000:1.0:1789762846.816462:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370116431745 time 370116526045 state 0x2 ref 0 00000100:00100000:3.0:1789762846.818262:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762846.818302:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015052288 00000100:00100000:1.0:1789762846.818311:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015052288 00000100:00100000:1.0:1789762846.818346:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370116431745 time 370116526045 state 0x0 ref 1 00000100:00100000:1.0:1789762846.818357:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370216526045 time 370116526045 state 0x1 ref 1 00000100:00100000:0.0:1789762846.912770:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370216526045 time 370216658892 state 0x0 ref 1 00000100:00100000:0.0:1789762846.912804:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015052288:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762846.912852:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049b9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.912869:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762846.912873:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762846.912879:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049b9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.912893:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049b9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.916249:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049b9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.916261:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049b9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.916269:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049b9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.916274:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a08000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762846.916279:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049b9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.916286:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049b9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.916302:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049b9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762846.916307:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049b9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762846.916366:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015052288:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3577us (98111us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762846.916375:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 883 00000100:00100000:0.0:1789762846.916390:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370216526045 time 370216658892 state 0x2 ref 0 00000100:00100000:2.0:1789762846.918109:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762846.918138:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015052416 00000100:00100000:0.0:1789762846.918148:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015052416 00000100:00100000:0.0:1789762846.918191:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370216526045 time 370216658892 state 0x0 ref 1 00000100:00100000:0.0:1789762846.918205:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370316658892 time 370216658892 state 0x1 ref 1 00000100:00100000:1.0:1789762847.012893:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370316658892 time 370316788196 state 0x0 ref 1 00000100:00100000:1.0:1789762847.012920:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015052416:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762847.012964:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.012979:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.012982:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.012987:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.012998:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.016575:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.016585:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.016593:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.016599:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762847.016604:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.016611:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.016627:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.016638:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762847.016698:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015052416:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3790us (98591us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762847.016705:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 884 00000100:00100000:1.0:1789762847.016713:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370316658892 time 370316788196 state 0x2 ref 0 00000100:00100000:0.0:1789762847.018477:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762847.018547:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015052544 00000100:00100000:1.0:1789762847.018556:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015052544 00000100:00100000:1.0:1789762847.018593:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370316658892 time 370316788196 state 0x0 ref 1 00000100:00100000:1.0:1789762847.018604:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370416788196 time 370316788196 state 0x1 ref 1 00000100:00100000:1.0:1789762847.112984:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370416788196 time 370416878726 state 0x0 ref 1 00000100:00100000:1.0:1789762847.113023:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84542300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015052544:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762847.113064:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.113079:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.113082:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.113116:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.113133:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.115941:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.115950:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.115957:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.115962:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762847.115965:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.115971:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.115984:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.115989:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049c7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762847.116060:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84542300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015052544:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3060us (97590us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762847.116067:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 885 00000100:00100000:1.0:1789762847.116075:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370416788196 time 370416878726 state 0x2 ref 0 00000100:00100000:3.0:1789762847.117850:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762847.117889:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015052672 00000100:00100000:1.0:1789762847.117897:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015052672 00000100:00100000:1.0:1789762847.117935:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370416788196 time 370416878726 state 0x0 ref 1 00000100:00100000:1.0:1789762847.117950:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370516878726 time 370416878726 state 0x1 ref 1 00000100:00100000:0.0:1789762847.213153:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370516878726 time 370517038622 state 0x0 ref 1 00000100:00100000:0.0:1789762847.213191:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015052672:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762847.213261:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049ce lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.213283:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762847.213289:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762847.213297:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049ce lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.213320:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049ce lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.216651:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049ce lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.216675:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049ce lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.216683:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049ce lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.216689:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a08000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762847.216693:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049ce lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.216699:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049ce lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.216716:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049ce lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.216722:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049ce lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762847.216782:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851baa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015052672:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3607us (98937us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762847.216790:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 886 00000100:00100000:0.0:1789762847.216799:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370516878726 time 370517038622 state 0x2 ref 0 00000100:00100000:2.0:1789762847.218528:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762847.218590:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015052800 00000100:00100000:0.0:1789762847.218610:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015052800 00000100:00100000:0.0:1789762847.218663:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370516878726 time 370517038622 state 0x0 ref 1 00000100:00100000:0.0:1789762847.218680:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370617038622 time 370517038622 state 0x1 ref 1 00000100:00100000:1.0:1789762847.313252:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370617038622 time 370617146594 state 0x0 ref 1 00000100:00100000:1.0:1789762847.313286:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015052800:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762847.313324:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049d5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.313337:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.313341:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.313346:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049d5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.313358:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049d5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.316433:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049d5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.316443:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049d5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.316449:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049d5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.316454:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762847.316458:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049d5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.316464:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049d5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.316477:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049d5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.316482:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049d5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762847.316544:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015052800:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3274us (98015us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762847.316556:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 887 00000100:00100000:1.0:1789762847.316568:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370617038622 time 370617146594 state 0x2 ref 0 00000100:00100000:3.0:1789762847.318366:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762847.318439:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015052928 00000100:00100000:1.0:1789762847.318447:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015052928 00000100:00100000:1.0:1789762847.318482:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370617038622 time 370617146594 state 0x0 ref 1 00000100:00100000:1.0:1789762847.318493:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370717146594 time 370617146594 state 0x1 ref 1 00000100:00100000:1.0:1789762847.413329:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370717146594 time 370717225797 state 0x0 ref 1 00000100:00100000:1.0:1789762847.413352:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015052928:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762847.413386:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049dc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.413407:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.413411:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.413416:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049dc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.413427:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049dc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.416348:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049dc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.416359:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049dc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.416367:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049dc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.416374:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762847.416377:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049dc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.416383:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049dc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.416399:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049dc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.416405:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049dc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762847.416466:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015052928:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3122us (98110us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762847.416472:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 888 00000100:00100000:1.0:1789762847.416480:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370717146594 time 370717225797 state 0x2 ref 0 00000100:00100000:3.0:1789762847.418270:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762847.418312:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015053056 00000100:00100000:1.0:1789762847.418321:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015053056 00000100:00100000:1.0:1789762847.418357:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370717146594 time 370717225797 state 0x0 ref 1 00000100:00100000:1.0:1789762847.418370:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370817225797 time 370717225797 state 0x1 ref 1 00000100:00100000:0.0:1789762847.513477:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370817225797 time 370817366529 state 0x0 ref 1 00000100:00100000:0.0:1789762847.513512:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015053056:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762847.513564:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049e3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.513583:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762847.513588:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762847.513594:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049e3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.513607:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049e3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.517361:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049e3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.517373:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049e3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.517381:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049e3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.517405:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a08000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762847.517410:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049e3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.517418:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049e3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.517436:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049e3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.517442:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049e3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762847.517502:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015053056:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4006us (99239us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762847.517509:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 889 00000100:00100000:0.0:1789762847.517518:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370817225797 time 370817366529 state 0x2 ref 0 00000100:00100000:2.0:1789762847.519486:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762847.519554:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015053184 00000100:00100000:0.0:1789762847.519565:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015053184 00000100:00100000:0.0:1789762847.519609:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370817225797 time 370817366529 state 0x0 ref 1 00000100:00100000:0.0:1789762847.519621:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370917366529 time 370817366529 state 0x1 ref 1 00000100:00100000:1.0:1789762847.613583:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370917366529 time 370917471574 state 0x0 ref 1 00000100:00100000:1.0:1789762847.613618:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015053184:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762847.613666:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ea lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.613684:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.613688:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.613694:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.613708:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.616787:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.616799:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ea lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.616807:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.616819:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762847.616823:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.616831:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.616847:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ea lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.616853:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ea lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762847.616913:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015053184:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3310us (97427us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762847.616921:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 890 00000100:00100000:1.0:1789762847.616930:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370917366529 time 370917471574 state 0x2 ref 0 00000100:00100000:0.0:1789762847.619261:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762847.619312:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015053312 00000100:00100000:1.0:1789762847.619322:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015053312 00000100:00100000:1.0:1789762847.619371:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 370917366529 time 370917471574 state 0x0 ref 1 00000100:00100000:1.0:1789762847.619383:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371017471574 time 370917471574 state 0x1 ref 1 00010000:00010000:1.0:1789762847.671235:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762847.713681:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371017471574 time 371017562961 state 0x0 ref 1 00000100:00100000:1.0:1789762847.713719:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015053312:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762847.713786:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049f1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.713809:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762847.713815:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.713823:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049f1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.713851:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049f1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.717082:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049f1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.717113:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049f1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.717122:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049f1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.717128:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762847.717133:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049f1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.717144:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049f1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.717161:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049f1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.717166:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049f1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762847.717226:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015053312:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3522us (97965us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762847.717234:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 891 00000100:00100000:1.0:1789762847.717243:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371017471574 time 371017562961 state 0x2 ref 0 00000100:00100000:0.0:1789762847.719258:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762847.719290:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015053440 00000100:00100000:1.0:1789762847.719300:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015053440 00000100:00100000:1.0:1789762847.719349:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371017471574 time 371017562961 state 0x0 ref 1 00000100:00100000:1.0:1789762847.719362:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371117562961 time 371017562961 state 0x1 ref 1 00000100:00100000:0.0:1789762847.813796:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371117562961 time 371117681904 state 0x0 ref 1 00000100:00100000:0.0:1789762847.813831:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015053440:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762847.813887:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049f8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.813909:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762847.813914:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762847.813921:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049f8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762847.813938:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049f8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762847.817584:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049f8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762847.817595:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049f8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762847.817604:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049f8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762847.817611:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a08000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762847.817615:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049f8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762847.817622:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049f8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762847.817640:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049f8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762847.817646:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b049f8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762847.817703:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015053440:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3887us (98449us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762847.817710:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 892 00000100:00100000:3.0:1789762847.817719:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371117562961 time 371117681904 state 0x2 ref 0 00000100:00100000:2.0:1789762847.819885:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762847.819937:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015053568 00000100:00100000:0.0:1789762847.819947:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015053568 00000100:00100000:0.0:1789762847.819996:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371117562961 time 371117681904 state 0x0 ref 1 00000100:00100000:0.0:1789762847.820024:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371217681904 time 371117681904 state 0x1 ref 1 00000100:00100000:1.0:1789762847.913924:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371217681904 time 371217812876 state 0x0 ref 1 00000100:00100000:1.0:1789762847.913956:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015053568:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762847.914019:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ff lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.914038:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.914047:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762847.914054:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.914069:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.917611:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.917622:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ff lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.917630:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.917635:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762847.917639:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.917646:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ff lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.917661:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ff lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762847.917671:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b049ff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762847.917729:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb8c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015053568:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3787us (97846us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762847.917737:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 893 00000100:00100000:1.0:1789762847.917746:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371217681904 time 371217812876 state 0x2 ref 0 00000100:00100000:2.0:1789762847.919683:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762847.919737:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015053696 00000100:00100000:1.0:1789762847.919749:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015053696 00000100:00100000:1.0:1789762847.919799:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371217681904 time 371217812876 state 0x0 ref 1 00000100:00100000:1.0:1789762847.919812:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371317812876 time 371217812876 state 0x1 ref 1 00000100:00100000:1.0:1789762848.013991:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371317812876 time 371317880804 state 0x0 ref 1 00000100:00100000:1.0:1789762848.014057:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015053696:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:2.0:1789762848.014081:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762848.014132:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a06 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.014154:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.014159:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.014167:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a06 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.014186:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a06 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.017419:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a06 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.017430:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a06 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.017438:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a06 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.017445:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762848.017449:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a06 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.017456:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a06 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.017473:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a06 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.017478:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a06 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762848.017540:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015053696:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3516us (97858us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762848.017548:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 894 00000100:00100000:1.0:1789762848.017556:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371317812876 time 371317880804 state 0x2 ref 0 00000100:00100000:2.0:1789762848.019456:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762848.019494:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015053824 00000100:00100000:1.0:1789762848.019504:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015053824 00000100:00100000:1.0:1789762848.019547:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371317812876 time 371317880804 state 0x0 ref 1 00000100:00100000:1.0:1789762848.019560:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371417880804 time 371317880804 state 0x1 ref 1 00000100:00100000:3.0:1789762848.114107:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371417880804 time 371417985764 state 0x0 ref 1 00000100:00100000:3.0:1789762848.114129:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015053824:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762848.114165:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a0d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.114179:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762848.114182:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762848.114187:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a0d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.114198:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a0d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.116863:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a0d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.116872:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a0d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.116879:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a0d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.116884:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762848.116887:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a0d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.116895:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a0d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.116907:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a0d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.116912:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a0d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762848.116958:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015053824:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2836us (97503us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762848.116966:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 895 00000100:00100000:3.0:1789762848.116973:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371417880804 time 371417985764 state 0x2 ref 0 00000100:00100000:2.0:1789762848.118889:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762848.118931:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015053952 00000100:00100000:0.0:1789762848.118939:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015053952 00000100:00100000:0.0:1789762848.118974:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371417880804 time 371417985764 state 0x0 ref 1 00000100:00100000:0.0:1789762848.118985:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371517985764 time 371417985764 state 0x1 ref 1 00000100:00100000:1.0:1789762848.214211:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371517985764 time 371518095264 state 0x0 ref 1 00000100:00100000:1.0:1789762848.214250:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015053952:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762848.214305:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a14 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.214326:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.214332:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.214339:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a14 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.214355:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a14 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.217762:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a14 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.217774:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a14 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.217785:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a14 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.217795:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762848.217800:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a14 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.217811:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a14 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.217836:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a14 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.217845:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a14 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762848.217919:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015053952:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3684us (99029us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762848.217931:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 896 00000100:00100000:1.0:1789762848.217944:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371517985764 time 371518095264 state 0x2 ref 0 00000100:00100000:0.0:1789762848.219965:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762848.220028:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015054080 00000100:00100000:1.0:1789762848.220040:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015054080 00000100:00100000:1.0:1789762848.220114:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371517985764 time 371518095264 state 0x0 ref 1 00000100:00100000:1.0:1789762848.220131:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371618095264 time 371518095264 state 0x1 ref 1 00000100:00100000:1.0:1789762848.314300:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371618095264 time 371618183329 state 0x0 ref 1 00000100:00100000:1.0:1789762848.314341:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015054080:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762848.314398:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a1b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.314420:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.314426:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.314434:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a1b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.314452:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a1b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.317899:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a1b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.317912:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a1b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.317923:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a1b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.317932:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762848.317937:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a1b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.317948:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a1b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.317969:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a1b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.317979:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a1b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762848.318070:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015054080:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3747us (98111us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762848.318083:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 897 00000100:00100000:1.0:1789762848.318143:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371618095264 time 371618183329 state 0x2 ref 0 00000100:00100000:0.0:1789762848.320057:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762848.320126:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015054208 00000100:00100000:1.0:1789762848.320138:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015054208 00000100:00100000:1.0:1789762848.320187:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371618095264 time 371618183329 state 0x0 ref 1 00000100:00100000:1.0:1789762848.320207:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371718183329 time 371618183329 state 0x1 ref 1 00000100:00100000:3.0:1789762848.414422:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371718183329 time 371718318673 state 0x0 ref 1 00000100:00100000:3.0:1789762848.414443:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015054208:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762848.414484:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a22 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.414512:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762848.414516:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762848.414521:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a22 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.414537:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a22 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.417795:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a22 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.417805:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a22 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.417811:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a22 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.417816:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762848.417820:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a22 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.417826:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a22 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.417840:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a22 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.417846:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a22 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762848.417895:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015054208:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3458us (97850us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762848.417902:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 898 00000100:00100000:3.0:1789762848.417910:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371718183329 time 371718318673 state 0x2 ref 0 00000100:00100000:2.0:1789762848.419844:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762848.419888:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015054336 00000100:00100000:0.0:1789762848.419896:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015054336 00000100:00100000:0.0:1789762848.419931:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371718183329 time 371718318673 state 0x0 ref 1 00000100:00100000:0.0:1789762848.419949:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371818318673 time 371718318673 state 0x1 ref 1 00000100:00100000:1.0:1789762848.514560:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371818318673 time 371818444046 state 0x0 ref 1 00000100:00100000:1.0:1789762848.514598:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015054336:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762848.514654:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a29 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.514676:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.514682:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.514690:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a29 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.514708:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a29 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.518039:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a29 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.518052:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a29 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.518062:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a29 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.518077:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762848.518082:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a29 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.518116:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a29 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.518137:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a29 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.518145:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a29 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762848.518217:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015054336:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3634us (98372us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762848.518231:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 899 00000100:00100000:1.0:1789762848.518243:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371818318673 time 371818444046 state 0x2 ref 0 00000100:00100000:3.0:1789762848.520554:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762848.520604:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015054464 00000100:00100000:1.0:1789762848.520616:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015054464 00000100:00100000:1.0:1789762848.520668:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371818318673 time 371818444046 state 0x0 ref 1 00000100:00100000:1.0:1789762848.520686:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371918444046 time 371818444046 state 0x1 ref 1 00000100:00100000:1.0:1789762848.614641:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371918444046 time 371918528093 state 0x0 ref 1 00000100:00100000:1.0:1789762848.614676:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015054464:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762848.614732:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a30 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.614753:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.614759:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.614766:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a30 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.614784:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a30 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.618431:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a30 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.618444:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a30 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.618456:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a30 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.618465:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762848.618471:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a30 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.618489:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a30 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.618511:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a30 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.618520:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a30 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762848.618588:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015054464:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3924us (98039us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762848.618598:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 900 00000100:00100000:1.0:1789762848.618608:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371918444046 time 371918528093 state 0x2 ref 0 00000100:00100000:3.0:1789762848.620893:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762848.620948:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015054592 00000100:00100000:1.0:1789762848.620959:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015054592 00000100:00100000:1.0:1789762848.621022:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 371918444046 time 371918528093 state 0x0 ref 1 00000100:00100000:1.0:1789762848.621039:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372018528093 time 371918528093 state 0x1 ref 1 00010000:00010000:0.0:1789762848.695262:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762848.714750:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372018528093 time 372018640973 state 0x0 ref 1 00000100:00100000:3.0:1789762848.714793:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015054592:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762848.714841:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a37 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.714859:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762848.714863:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762848.714868:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a37 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.714883:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a37 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.718377:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a37 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.718391:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a37 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.718397:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a37 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.718402:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762848.718405:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a37 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.718412:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a37 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.718424:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a37 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762848.718430:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a37 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762848.718476:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015054592:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3707us (97589us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762848.718482:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 901 00000100:00100000:3.0:1789762848.718489:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372018528093 time 372018640973 state 0x2 ref 0 00000100:00100000:3.0:1789762848.721029:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762848.721084:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015054720 00000100:00100000:0.0:1789762848.721107:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015054720 00000100:00100000:0.0:1789762848.721144:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372018528093 time 372018640973 state 0x0 ref 1 00000100:00100000:0.0:1789762848.721155:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372118640973 time 372018640973 state 0x1 ref 1 00000100:00100000:1.0:1789762848.814844:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372118640973 time 372118740298 state 0x0 ref 1 00000100:00100000:1.0:1789762848.814867:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015054720:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762848.814903:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a3e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.814918:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.814921:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.814926:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a3e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.814937:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a3e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.818041:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a3e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.818057:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a3e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.818068:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a3e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.818074:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762848.818079:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a3e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.818102:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a3e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.818116:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a3e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.818127:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a3e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762848.818175:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851ba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015054720:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3316us (97146us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762848.818181:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 902 00000100:00100000:1.0:1789762848.818189:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372118640973 time 372118740298 state 0x2 ref 0 00000100:00100000:0.0:1789762848.820158:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762848.820191:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015054848 00000100:00100000:1.0:1789762848.820199:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015054848 00000100:00100000:1.0:1789762848.820232:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372118640973 time 372118740298 state 0x0 ref 1 00000100:00100000:1.0:1789762848.820243:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372218740298 time 372118740298 state 0x1 ref 1 00000100:00100000:3.0:1789762848.866275:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762848.866308:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762848.866329:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762848.866332:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015054976 00000100:00100000:1.0:1789762848.866343:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015054976 00000100:00100000:0.0:1789762848.866359:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015055104 00000100:00100000:3.0:1789762848.866360:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762848.866367:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015055104 00000100:00100000:2.0:1789762848.866392:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015055232 00000100:00100000:3.0:1789762848.866396:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015055360 00000100:00100000:1.0:1789762848.866399:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 318 00000100:00100000:2.0:1789762848.866399:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015055232 00000100:00100000:3.0:1789762848.866403:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015055360 00000100:00100000:1.0:1789762848.866414:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85055850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015054976:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762848.866421:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 283 00000100:00100000:0.0:1789762848.866433:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2366:x1876702015055104:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762848.866456:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 284 00000100:00100000:2.0:1789762848.866462:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 143 00000100:00100000:3.0:1789762848.866468:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851ba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015055360:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762848.866476:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015055232:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762848.866482:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85055850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015054976:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 70us (211us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762848.866487:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2366:x1876702015055104:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 54us (173us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762848.866494:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 318 00000100:00100000:3.0:1789762848.866504:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851ba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015055360:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 39us (146us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762848.866509:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 283 00000100:00100000:3.0:1789762848.866522:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 284 00000100:00100000:2.0:1789762848.866539:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015055232:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 67us (207us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762848.866545:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789762848.866551:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762848.866551:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 143 00000100:00100000:2.0:1789762848.887229:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789762848.887248:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022175232:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762848.887306:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762848.887359:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022175232 00000100:00100000:3.0:1789762848.887367:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022175232 00000100:00080000:1.0:1789762848.887409:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762848.887441:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 119 00000100:00100000:2.0:1789762848.887454:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022175232:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762848.887485:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762848.887536:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb17100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022175232:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762848.887545:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 119 00000100:00100000:0.0:1789762848.887567:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022175232:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762848.914892:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372218740298 time 372218791291 state 0x0 ref 1 00000100:00100000:1.0:1789762848.914909:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84543480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015054848:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762848.914938:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a45 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.914951:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.914954:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762848.914959:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a45 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762848.914970:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a45 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762848.917808:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a45 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762848.917819:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a45 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762848.917826:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a45 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762848.917831:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762848.917834:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a45 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762848.917840:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a45 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762848.917861:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a45 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762848.917867:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04a45 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762848.917917:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015054848:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3012us (97760us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762848.917924:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 903 00000100:00100000:2.0:1789762848.917932:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372218740298 time 372218791291 state 0x2 ref 0 00000100:00100000:3.0:1789762848.919918:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762848.919965:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015055488 00000100:00100000:1.0:1789762848.919973:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015055488 00000100:00100000:1.0:1789762848.920018:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372218740298 time 372218791291 state 0x0 ref 1 00000100:00100000:1.0:1789762848.920029:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372318791291 time 372218791291 state 0x1 ref 1 00000100:00100000:3.0:1789762849.014990:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372318791291 time 372318883806 state 0x0 ref 1 00000100:00100000:3.0:1789762849.015027:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015055488:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762849.015065:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a4c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.015078:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762849.015082:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762849.015108:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a4c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.015120:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a4c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.017913:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a4c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.017925:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a4c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.017932:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a4c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.017938:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762849.017942:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a4c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.017948:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a4c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.017963:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a4c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.017968:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a81f1fd/0x416193e012b04a4c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762849.018024:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015055488:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3006us (98106us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762849.018036:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 904 00000100:00100000:3.0:1789762849.018048:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372318791291 time 372318883806 state 0x2 ref 0 00000100:00100000:3.0:1789762849.019929:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762849.019986:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015055616 00000100:00100000:2.0:1789762849.020006:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015055616 00000100:00100000:2.0:1789762849.020048:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372318791291 time 372318883806 state 0x0 ref 1 00000100:00100000:2.0:1789762849.020062:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372418883806 time 372318883806 state 0x1 ref 1 00000100:00100000:1.0:1789762849.115079:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372418883806 time 372418974776 state 0x0 ref 1 00000100:00100000:1.0:1789762849.115116:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015055616:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762849.115154:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a53 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.115172:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762849.115175:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762849.115180:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a53 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.115191:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a53 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.118226:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a53 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.118238:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a53 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.118248:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a53 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.118257:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762849.118262:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a53 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.118274:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a53 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.118290:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a53 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.118300:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a53 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762849.118347:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015055616:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3239us (98419us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762849.118354:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 905 00000100:00100000:1.0:1789762849.118362:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372418883806 time 372418974776 state 0x2 ref 0 00000100:00100000:1.0:1789762849.120204:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762849.120237:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015055744 00000100:00100000:3.0:1789762849.120245:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015055744 00000100:00100000:3.0:1789762849.120281:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372418883806 time 372418974776 state 0x0 ref 1 00000100:00100000:3.0:1789762849.120291:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372518974776 time 372418974776 state 0x1 ref 1 00000100:00100000:2.0:1789762849.215162:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372518974776 time 372519055518 state 0x0 ref 1 00000100:00100000:2.0:1789762849.215189:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015055744:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762849.215227:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04a5a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.215244:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762849.215261:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762849.215268:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04a5a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.215283:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04a5a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.218452:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04a5a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.218464:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04a5a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.218474:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04a5a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.218481:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762849.218486:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04a5a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.218495:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04a5a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.218521:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04a5a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.218530:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04a5a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762849.218594:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015055744:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3414us (98389us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762849.218604:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 906 00000100:00100000:2.0:1789762849.218614:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372518974776 time 372519055518 state 0x2 ref 0 00000100:00100000:1.0:1789762849.220459:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762849.220512:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015055872 00000100:00100000:2.0:1789762849.220521:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015055872 00000100:00100000:2.0:1789762849.220563:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372518974776 time 372519055518 state 0x0 ref 1 00000100:00100000:2.0:1789762849.220577:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372619055518 time 372519055518 state 0x1 ref 1 00000100:00100000:3.0:1789762849.315257:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372619055518 time 372619152444 state 0x0 ref 1 00000100:00100000:3.0:1789762849.315281:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015055872:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762849.315319:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a61 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.315333:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762849.315337:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762849.315342:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a61 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.315354:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a61 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.317831:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a61 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.317846:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a61 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.317853:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a61 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.317858:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762849.317861:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a61 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.317868:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a61 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.317880:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a61 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.317885:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a61 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762849.317932:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015055872:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2660us (97475us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762849.317939:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 907 00000100:00100000:3.0:1789762849.317946:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372619055518 time 372619152444 state 0x2 ref 0 00000100:00100000:3.0:1789762849.319782:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762849.319821:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015056000 00000100:00100000:1.0:1789762849.319830:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015056000 00000100:00100000:1.0:1789762849.319867:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372619055518 time 372619152444 state 0x0 ref 1 00000100:00100000:1.0:1789762849.319877:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372719152444 time 372619152444 state 0x1 ref 1 00000100:00100000:0.0:1789762849.399245:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022175616:0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.399245:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d84542d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022175488:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762849.399283:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022175744:0@lo:400:kworker.0 00000100:00100000:3.0:1789762849.399302:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022175360:0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.399327:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762849.399328:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762849.399349:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762849.399377:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022175488 00000100:00100000:2.0:1789762849.399387:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022175488 00000100:00100000:1.0:1789762849.399440:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4bcd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022176000:0@lo:400:kworker.0 00000100:00100000:3.0:1789762849.399444:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022175872:0@lo:400:kworker.0 00000100:00100000:0.0:1789762849.399449:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022175616 00000100:00100000:0.0:1789762849.399455:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022175616 00000100:00100000:2.0:1789762849.399462:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 120 00000100:00100000:2.0:1789762849.399476:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022175488:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762849.399484:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762849.399493:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:3.0:1789762849.399498:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762849.399506:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84543b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022175616:12345-0@lo:400:kworker.0 00002000:00000024:2.0:1789762849.399518:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762849.399536:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022176000 00000100:00100000:1.0:1789762849.399542:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022176000 00000100:00100000:3.0:1789762849.399550:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022175360 00000100:00100000:3.0:1789762849.399555:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022175360 00000100:00100000:0.0:1789762849.399566:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022175616:12345-0@lo:400:kworker.0 Request processed in 61us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762849.399576:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:1.0:1789762849.399588:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:2.0:1789762849.399594:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022175488:12345-0@lo:13:osp-pre-1-0.0 Request processed in 119us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762849.399602:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022176000:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762849.399603:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022175872 00000100:00100000:3.0:1789762849.399606:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 319 00000100:00100000:2.0:1789762849.399610:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 120 00000100:00100000:0.0:1789762849.399611:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022175872 00000100:00100000:3.0:1789762849.399625:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85057c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022175360:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762849.399650:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762849.399652:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00080000:2.0:1789762849.399659:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762849.399667:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022176000:12345-0@lo:400:kworker.0 Request processed in 67us (184us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762849.399676:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022175872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.399677:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00100000:2.0:1789762849.399682:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762849.399688:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85057c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022175360:12345-0@lo:400:kworker.0 Request processed in 66us (337us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762849.399696:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 319 00000100:00100000:1.0:1789762849.399720:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022175744 00000100:00100000:1.0:1789762849.399725:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022175744 00000100:00100000:0.0:1789762849.399732:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022175872:12345-0@lo:400:kworker.0 Request processed in 70us (239us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762849.399740:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:0.0:1789762849.399759:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:2.0:1789762849.399769:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4bcd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022176000:0@lo:400:kworker.0 00000100:00100000:0.0:1789762849.399770:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022175744:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762849.399776:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022175360:0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.399795:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022175616:0@lo:400:kworker.0 00000100:00100000:3.0:1789762849.399800:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022175872:0@lo:400:kworker.0 00000100:00100000:2.0:1789762849.399815:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d84542d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022175488:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762849.399823:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022175744:12345-0@lo:400:kworker.0 Request processed in 55us (146us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762849.399833:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00080000:3.0:1789762849.399843:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762849.399866:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022175744:0@lo:400:kworker.0 00000100:00100000:1.0:1789762849.415290:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372719152444 time 372719191248 state 0x0 ref 1 00000100:00100000:1.0:1789762849.415305:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015056000:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762849.415332:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a68 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.415346:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762849.415350:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762849.415354:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a68 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.415365:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a68 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.418447:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a68 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.418461:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a68 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.418478:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a68 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.418488:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762849.418493:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a68 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.418505:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a68 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.418520:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a68 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.418526:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a68 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762849.418571:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015056000:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3269us (98788us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762849.418577:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 908 00000100:00100000:1.0:1789762849.418584:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372719152444 time 372719191248 state 0x2 ref 0 00000100:00100000:3.0:1789762849.420385:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762849.420432:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015056128 00000100:00100000:1.0:1789762849.420438:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015056128 00000100:00100000:1.0:1789762849.420473:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372719152444 time 372719191248 state 0x0 ref 1 00000100:00100000:1.0:1789762849.420493:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372819191248 time 372719191248 state 0x1 ref 1 00000100:00100000:2.0:1789762849.515385:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372819191248 time 372819276881 state 0x0 ref 1 00000100:00100000:2.0:1789762849.515413:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015056128:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762849.515453:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04a6f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.515470:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762849.515474:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762849.515480:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04a6f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.515494:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04a6f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.518733:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04a6f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.518746:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04a6f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.518755:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04a6f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.518781:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762849.518786:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04a6f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.518796:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04a6f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.518813:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04a6f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.518821:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b04a6f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762849.518878:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015056128:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3474us (98492us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762849.518888:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 909 00000100:00100000:2.0:1789762849.518897:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372819191248 time 372819276881 state 0x2 ref 0 00000100:00100000:2.0:1789762849.520818:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762849.520894:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015056256 00000100:00100000:1.0:1789762849.520903:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015056256 00000100:00100000:1.0:1789762849.520950:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372819191248 time 372819276881 state 0x0 ref 1 00000100:00100000:1.0:1789762849.520961:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372919276881 time 372819276881 state 0x1 ref 1 00000100:00100000:3.0:1789762849.615467:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372919276881 time 372919363526 state 0x0 ref 1 00000100:00100000:3.0:1789762849.615489:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015056256:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762849.615525:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a76 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.615539:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762849.615542:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762849.615546:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a76 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.615563:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a76 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.618432:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a76 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.618442:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a76 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.618449:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a76 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.618454:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762849.618457:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a76 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.618464:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a76 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.618479:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a76 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.618485:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04a76 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762849.618534:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015056256:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3052us (97716us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762849.618541:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 910 00000100:00100000:3.0:1789762849.618549:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372919276881 time 372919363526 state 0x2 ref 0 00000100:00100000:2.0:1789762849.620294:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762849.620347:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015056384 00000100:00100000:1.0:1789762849.620355:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015056384 00000100:00100000:1.0:1789762849.620392:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 372919276881 time 372919363526 state 0x0 ref 1 00000100:00100000:1.0:1789762849.620402:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373019363526 time 372919363526 state 0x1 ref 1 00000100:00100000:1.0:1789762849.715517:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373019363526 time 373019413916 state 0x0 ref 1 00000100:00100000:1.0:1789762849.715540:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015056384:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762849.715577:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a7d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.715592:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762849.715596:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762849.715600:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a7d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.715616:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a7d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.718868:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a7d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.718879:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a7d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.718886:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a7d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.718890:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762849.718894:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a7d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.718900:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a7d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.718924:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a7d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762849.718933:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a7d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762849.719007:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb17480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015056384:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3473us (98712us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762849.719017:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 911 00000100:00100000:1.0:1789762849.719027:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373019363526 time 373019413916 state 0x2 ref 0 00010000:00010000:0.0:1789762849.719203:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762849.720877:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762849.720930:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015056512 00000100:00100000:1.0:1789762849.720940:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015056512 00000100:00100000:1.0:1789762849.720980:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373019363526 time 373019413916 state 0x0 ref 1 00000100:00100000:1.0:1789762849.721008:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373119413916 time 373019413916 state 0x1 ref 1 00000100:00100000:2.0:1789762849.815599:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373119413916 time 373119495096 state 0x0 ref 1 00000100:00100000:2.0:1789762849.815624:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015056512:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762849.815668:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a84 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.815691:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762849.815695:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762849.815700:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a84 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.815712:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a84 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.818254:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a84 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.818263:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a84 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.818270:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a84 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.818275:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a09e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762849.818279:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a84 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.818285:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a84 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.818301:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a84 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762849.818317:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a84 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762849.818361:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015056512:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2746us (97484us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762849.818368:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 912 00000100:00100000:2.0:1789762849.818376:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373119413916 time 373119495096 state 0x2 ref 0 00000100:00100000:3.0:1789762849.820244:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762849.820286:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015056640 00000100:00100000:1.0:1789762849.820294:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015056640 00000100:00100000:1.0:1789762849.820339:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373119413916 time 373119495096 state 0x0 ref 1 00000100:00100000:1.0:1789762849.820350:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373219495096 time 373119495096 state 0x1 ref 1 00000100:00100000:3.0:1789762849.915715:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373219495096 time 373219610629 state 0x0 ref 1 00000100:00100000:3.0:1789762849.915740:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015056640:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762849.915776:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b04a8b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.915790:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762849.915793:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762849.915798:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b04a8b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.915809:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b04a8b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.919343:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b04a8b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.919359:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b04a8b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.919366:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b04a8b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.919374:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762849.919378:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b04a8b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.919388:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b04a8b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.919406:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b04a8b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762849.919416:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b04a8b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762849.919464:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015056640:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3733us (99219us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762849.919475:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 913 00000100:00100000:3.0:1789762849.919486:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373219495096 time 373219610629 state 0x2 ref 0 00000100:00100000:3.0:1789762849.921507:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762849.921555:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015056768 00000100:00100000:1.0:1789762849.921564:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015056768 00000100:00100000:1.0:1789762849.921608:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373219495096 time 373219610629 state 0x0 ref 1 00000100:00100000:1.0:1789762849.921625:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373319610629 time 373219610629 state 0x1 ref 1 00000100:00100000:1.0:1789762850.015787:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373319610629 time 373319678995 state 0x0 ref 1 00000100:00100000:1.0:1789762850.015828:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015056768:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:3.0:1789762850.015854:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762850.015873:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a92 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.015891:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762850.015896:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762850.015903:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a92 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.015926:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a92 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.018697:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a92 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.018710:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a92 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.018720:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a92 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.018730:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762850.018735:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a92 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.018746:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a92 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.018764:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a92 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.018773:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b04a92 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762850.018829:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015056768:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3022us (97328us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762850.018840:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 914 00000100:00100000:1.0:1789762850.018850:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373319610629 time 373319678995 state 0x2 ref 0 00000100:00100000:3.0:1789762850.020817:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762850.020860:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015056896 00000100:00100000:1.0:1789762850.020869:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015056896 00000100:00100000:1.0:1789762850.020909:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373319610629 time 373319678995 state 0x0 ref 1 00000100:00100000:1.0:1789762850.020924:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373419678995 time 373319678995 state 0x1 ref 1 00000100:00100000:2.0:1789762850.115853:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373419678995 time 373419748410 state 0x0 ref 1 00000100:00100000:2.0:1789762850.115875:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015056896:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762850.115911:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a99 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762850.115925:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762850.115929:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762850.115933:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a99 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762850.115944:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a99 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762850.118769:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a99 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762850.118792:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a99 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762850.118800:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a99 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762850.118805:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a09e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762850.118808:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a99 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762850.118815:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a99 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762850.118828:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a99 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762850.118833:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000087d49830/0x416193e012b04a99 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762850.118878:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015056896:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3011us (98062us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762850.118885:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 915 00000100:00100000:2.0:1789762850.118892:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373419678995 time 373419748410 state 0x2 ref 0 00000100:00100000:2.0:1789762850.120725:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762850.120783:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015057024 00000100:00100000:1.0:1789762850.120793:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015057024 00000100:00100000:1.0:1789762850.120839:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373419678995 time 373419748410 state 0x0 ref 1 00000100:00100000:1.0:1789762850.120851:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373519748410 time 373419748410 state 0x1 ref 1 00000100:00100000:3.0:1789762850.215969:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373519748410 time 373519865921 state 0x0 ref 1 00000100:00100000:3.0:1789762850.216006:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015057024:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762850.216044:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04aa0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.216057:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762850.216061:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762850.216065:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04aa0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.216076:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04aa0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.219133:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04aa0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.219143:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04aa0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.219149:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04aa0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.219155:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87a84200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762850.219158:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04aa0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.219166:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04aa0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.219181:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04aa0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.219187:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04aa0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762850.219232:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015057024:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3248us (98508us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762850.219240:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 916 00000100:00100000:3.0:1789762850.219247:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373519748410 time 373519865921 state 0x2 ref 0 00000100:00100000:0.0:1789762850.221207:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762850.221255:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015057152 00000100:00100000:1.0:1789762850.221265:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015057152 00000100:00100000:1.0:1789762850.221300:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373519748410 time 373519865921 state 0x0 ref 1 00000100:00100000:1.0:1789762850.221310:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373619865921 time 373519865921 state 0x1 ref 1 00000100:00100000:1.0:1789762850.316078:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373619865921 time 373619975035 state 0x0 ref 1 00000100:00100000:1.0:1789762850.316118:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015057152:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762850.316154:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04aa7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.316168:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762850.316171:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762850.316176:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04aa7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.316187:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04aa7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.319322:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04aa7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.319333:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04aa7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.319340:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04aa7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.319347:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762850.319350:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04aa7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.319357:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04aa7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.319370:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04aa7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.319375:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04aa7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762850.319423:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015057152:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3313us (98216us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762850.319430:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 917 00000100:00100000:1.0:1789762850.319437:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373619865921 time 373619975035 state 0x2 ref 0 00000100:00100000:1.0:1789762850.321227:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762850.321275:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015057280 00000100:00100000:3.0:1789762850.321284:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015057280 00000100:00100000:3.0:1789762850.321321:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373619865921 time 373619975035 state 0x0 ref 1 00000100:00100000:3.0:1789762850.321331:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373719975035 time 373619975035 state 0x1 ref 1 00000100:00100000:2.0:1789762850.416200:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373719975035 time 373720095341 state 0x0 ref 1 00000100:00100000:2.0:1789762850.416226:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015057280:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762850.416263:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b04aae lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762850.416277:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762850.416280:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762850.416285:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b04aae lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762850.416295:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b04aae lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.419260:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b04aae lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.419271:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b04aae lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.419284:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b04aae lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.419290:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762850.419293:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b04aae lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.419299:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b04aae lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.419314:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b04aae lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.419319:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b04aae lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762850.419376:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015057280:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3159us (98148us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762850.419388:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 918 00000100:00100000:1.0:1789762850.419400:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373719975035 time 373720095341 state 0x2 ref 0 00000100:00100000:1.0:1789762850.421268:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762850.421355:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015057408 00000100:00100000:2.0:1789762850.421364:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015057408 00000100:00100000:2.0:1789762850.421400:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373719975035 time 373720095341 state 0x0 ref 1 00000100:00100000:2.0:1789762850.421411:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373820095341 time 373720095341 state 0x1 ref 1 00000100:00100000:3.0:1789762850.516315:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373820095341 time 373820209936 state 0x0 ref 1 00000100:00100000:3.0:1789762850.516339:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015057408:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762850.516377:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04ab5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.516390:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762850.516394:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762850.516399:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04ab5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.516410:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04ab5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.519219:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04ab5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.519230:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04ab5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.519237:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04ab5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.519242:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87a84200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762850.519245:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04ab5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.519251:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04ab5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.519264:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04ab5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.519274:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005944a46e/0x416193e012b04ab5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762850.519328:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015057408:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2999us (98059us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762850.519335:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 919 00000100:00100000:3.0:1789762850.519343:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373820095341 time 373820209936 state 0x2 ref 0 00000100:00100000:0.0:1789762850.521256:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762850.521301:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015057536 00000100:00100000:1.0:1789762850.521310:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015057536 00000100:00100000:1.0:1789762850.521346:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373820095341 time 373820209936 state 0x0 ref 1 00000100:00100000:1.0:1789762850.521358:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373920209936 time 373820209936 state 0x1 ref 1 00000100:00100000:1.0:1789762850.616363:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373920209936 time 373920259705 state 0x0 ref 1 00000100:00100000:1.0:1789762850.616386:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84542a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015057536:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762850.616423:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04abc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.616438:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762850.616441:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762850.616446:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04abc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.616457:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04abc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.619393:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04abc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.619404:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04abc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.619411:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04abc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.619421:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762850.619425:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04abc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.619431:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04abc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.619446:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04abc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.619454:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04abc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762850.619516:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84542a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015057536:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3136us (98259us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762850.619524:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 920 00000100:00100000:1.0:1789762850.619531:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373920209936 time 373920259705 state 0x2 ref 0 00000100:00100000:2.0:1789762850.621451:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762850.621491:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015057664 00000100:00100000:3.0:1789762850.621499:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015057664 00000100:00100000:3.0:1789762850.621533:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 373920209936 time 373920259705 state 0x0 ref 1 00000100:00100000:3.0:1789762850.621543:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374020259705 time 373920259705 state 0x1 ref 1 00000100:00100000:1.0:1789762850.716442:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374020259705 time 374020339418 state 0x0 ref 1 00000100:00100000:1.0:1789762850.716471:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015057664:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762850.716513:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ac3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.716532:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762850.716535:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762850.716539:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ac3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.716552:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ac3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.719555:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ac3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.719575:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ac3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.719585:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ac3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.719592:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762850.719596:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ac3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.719602:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ac3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.719616:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ac3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.719624:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ac3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762850.719683:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015057664:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3221us (98239us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762850.719690:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 921 00000100:00100000:1.0:1789762850.719698:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374020259705 time 374020339418 state 0x2 ref 0 00000100:00100000:3.0:1789762850.721725:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762850.721771:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015057792 00000100:00100000:2.0:1789762850.721780:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015057792 00000100:00100000:2.0:1789762850.721819:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374020259705 time 374020339418 state 0x0 ref 1 00000100:00100000:2.0:1789762850.721830:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374120339418 time 374020339418 state 0x1 ref 1 00010000:00010000:0.0:1789762850.743191:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762850.816547:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374120339418 time 374120441195 state 0x0 ref 1 00000100:00100000:3.0:1789762850.816575:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015057792:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762850.816615:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04aca lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.816632:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762850.816636:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762850.816643:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04aca lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.816658:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04aca lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.819476:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04aca lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.819489:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04aca lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.819499:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04aca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.819507:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762850.819513:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04aca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.819523:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04aca lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.819539:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04aca lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762850.819549:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04aca lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762850.819610:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015057792:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3044us (97885us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762850.819619:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 922 00000100:00100000:3.0:1789762850.819630:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374120339418 time 374120441195 state 0x2 ref 0 00000100:00100000:3.0:1789762850.821557:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762850.821610:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015057920 00000100:00100000:1.0:1789762850.821618:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015057920 00000100:00100000:1.0:1789762850.821655:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374120339418 time 374120441195 state 0x0 ref 1 00000100:00100000:1.0:1789762850.821666:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374220441195 time 374120441195 state 0x1 ref 1 00000100:00100000:1.0:1789762850.916615:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374220441195 time 374220509571 state 0x0 ref 1 00000100:00100000:1.0:1789762850.916641:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015057920:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762850.916677:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.916691:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762850.916694:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762850.916699:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.916714:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.919255:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.919266:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.919273:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.919277:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762850.919281:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.919288:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.919301:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762850.919306:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762850.919367:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015057920:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2735us (97811us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762850.919374:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 923 00000100:00100000:1.0:1789762850.919381:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374220441195 time 374220509571 state 0x2 ref 0 00000100:00100000:3.0:1789762850.921259:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762850.921294:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015058048 00000100:00100000:1.0:1789762850.921302:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015058048 00000100:00100000:1.0:1789762850.921335:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374220441195 time 374220509571 state 0x0 ref 1 00000100:00100000:1.0:1789762850.921346:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374320509571 time 374220509571 state 0x1 ref 1 00000100:00100000:1.0:1789762851.016671:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374320509571 time 374320567340 state 0x0 ref 1 00000100:00100000:1.0:1789762851.016694:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015058048:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762851.016731:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.016744:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762851.016748:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762851.016753:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.016763:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.020130:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.020141:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.020149:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.020154:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762851.020158:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.020164:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.020178:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.020183:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b04ad8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762851.020230:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015058048:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3545us (98972us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762851.020237:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 924 00000100:00100000:1.0:1789762851.020245:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374320509571 time 374320567340 state 0x2 ref 0 00000100:00100000:3.0:1789762851.022107:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762851.022149:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015058176 00000100:00100000:2.0:1789762851.022158:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015058176 00000100:00100000:2.0:1789762851.022195:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374320509571 time 374320567340 state 0x0 ref 1 00000100:00100000:2.0:1789762851.022206:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374420567340 time 374320567340 state 0x1 ref 1 00000100:00100000:3.0:1789762851.116787:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374420567340 time 374420680233 state 0x0 ref 1 00000100:00100000:3.0:1789762851.116813:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015058176:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762851.116851:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04adf lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762851.116869:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762851.116874:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762851.116881:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04adf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762851.116895:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04adf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.120197:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04adf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.120232:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04adf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.120244:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04adf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.120253:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762851.120258:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04adf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.120267:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04adf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.120282:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04adf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.120288:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b04adf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762851.120342:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015058176:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3536us (98257us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762851.120353:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 925 00000100:00100000:0.0:1789762851.120365:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374420567340 time 374420680233 state 0x2 ref 0 00000100:00100000:2.0:1789762851.143385:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762851.143449:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015058304 00000100:00100000:0.0:1789762851.143473:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015058304 00000100:00100000:0.0:1789762851.143535:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 28 00000100:00100000:0.0:1789762851.143546:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:5592:x1876702015058304:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762851.143558:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762851.143594:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002fbc90b8/0x416193e012b04823 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6be expref: 6 pid: 6881 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.143615:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762851.143665:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:5592:x1876702015058304:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 122us (287us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762851.143670:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 28 00000100:00100000:3.0:1789762851.151587:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762851.151654:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015058432 00000100:00100000:0.0:1789762851.151659:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015058432 00000100:00080000:3.0:1789762851.151705:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762851.151708:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 302 00000100:00100000:0.0:1789762851.151718:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:12244:x1876702015058432:12345-192.168.203.48@tcp:101:bash.0 00010000:00010000:0.0:1789762851.151731:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762851.151745:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04ae6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6c5 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.151792:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04aed lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.151802:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04aed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.151810:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04aed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762851.151835:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00000004:00200000:0.0:1789762851.151844:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762851.151860:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04aed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.151878:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04ae6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6c5 expref: 11 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.151890:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04aed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6c5 expref: 10 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.151904:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981e9272c600) 00010000:00010000:0.0:1789762851.151906:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762851.151909:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86184a00, rc 0) 00000100:00100000:0.0:1789762851.151938:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:12244:x1876702015058432:12345-192.168.203.48@tcp:101:bash.0 Request processed in 222us (352us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762851.151943:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 302 00000100:00100000:3.0:1789762851.155378:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762851.155423:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015058688 00000100:00100000:1.0:1789762851.155429:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015058688 00000100:00100000:1.0:1789762851.155459:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 303 00000100:00100000:1.0:1789762851.155466:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14329:x1876702015058688:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:1.0:1789762851.155474:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762851.155483:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b04af4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6cc expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762851.155524:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00010000:00010000:1.0:1789762851.155642:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b04af4 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6cc expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.155650:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:1.0:1789762851.155652:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789762851.155655:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5c00/0x416193e012b04af4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6cc expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.155663:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb5c00, rc 0) 00000100:00100000:1.0:1789762851.155691:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bced80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14329:x1876702015058688:12345-192.168.203.48@tcp:101:rm.0 Request processed in 226us (314us total) trans 4294967555 rc 301/301 00000100:00100000:1.0:1789762851.155696:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 303 00000100:00100000:2.0:1789762851.157150:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762851.157184:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015058816 00000100:00100000:1.0:1789762851.157189:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015058816 00000100:00100000:1.0:1789762851.157218:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 95 00000100:00100000:1.0:1789762851.157226:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14329:x1876702015058816:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:1.0:1789762851.159204:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14329:x1876702015058816:12345-192.168.203.48@tcp:37:rm.0 Request processed in 1977us (2053us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762851.159210:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 95 00000100:00100000:3.0:1789762851.160393:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762851.160454:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015059072 00000100:00100000:1.0:1789762851.160460:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015059072 00000100:00100000:1.0:1789762851.160489:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 96 00000100:00100000:1.0:1789762851.160495:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14329:x1876702015059072:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:1.0:1789762851.160585:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14329:x1876702015059072:12345-192.168.203.48@tcp:35:rm.0 Request processed in 90us (194us total) trans 4294967556 rc 0/0 00000100:00100000:1.0:1789762851.160591:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 96 00000100:00100000:2.0:1789762851.161399:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762851.161433:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015059328 00000100:00100000:0.0:1789762851.161439:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015059328 00000100:00100000:0.0:1789762851.161467:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 304 00000100:00100000:0.0:1789762851.161474:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14329:x1876702015059328:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:0.0:1789762851.161483:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762851.161491:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185800/0x416193e012b04b02 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6d3 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.161526:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b04b09 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.161534:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b04b09 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.161550:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b04b09 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762851.161557:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00010000:00010000:0.0:1789762851.161608:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185800/0x416193e012b04b02 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6d3 expref: 15 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.161625:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b04b09 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6d3 expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.161633:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981e9272c600) 00010000:00010000:0.0:1789762851.161636:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1789762851.161639:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86184200, rc 0) 00000100:00100000:0.0:1789762851.161675:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb17800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14329:x1876702015059328:12345-192.168.203.48@tcp:101:rm.0 Request processed in 201us (277us total) trans 4294967557 rc 0/0 00000100:00100000:0.0:1789762851.161680:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 304 00000100:00100000:3.0:1789762851.162472:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762851.162508:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015059456 00000100:00100000:0.0:1789762851.162513:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015059456 00000100:00100000:0.0:1789762851.162537:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 305 00000100:00100000:0.0:1789762851.162542:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14329:x1876702015059456:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:0.0:1789762851.162569:0:8165:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7603224576 00000100:00100000:0.0:1789762851.162599:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14329:x1876702015059456:12345-192.168.203.48@tcp:41:rm.0 Request processed in 57us (128us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762851.162607:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 305 00000100:00100000:3.0:1789762851.163330:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762851.163359:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015059712 00000100:00100000:1.0:1789762851.163364:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015059712 00000100:00100000:1.0:1789762851.163392:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 97 00000100:00100000:1.0:1789762851.163401:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14329:x1876702015059712:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:1.0:1789762851.163566:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:14329:x1876702015059712:12345-192.168.203.48@tcp:35:rm.0 Request processed in 165us (235us total) trans 4294967558 rc 0/0 00000100:00100000:1.0:1789762851.163576:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 97 00000100:00100000:2.0:1789762851.164254:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762851.164282:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015059968 00000100:00100000:1.0:1789762851.164287:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015059968 00000100:00100000:1.0:1789762851.164310:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 306 00000100:00100000:1.0:1789762851.164315:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:14329:x1876702015059968:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:1.0:1789762851.164337:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1789762851.164349:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87c1a800/0x416193e012b04815 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x32:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6b7 expref: 15 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.164366:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87cbe200/0x416193e012b04528 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6b0 expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.164373:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff981ea0aa3500) 00010000:00010000:1.0:1789762851.164381:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04aed lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6c5 expref: 13 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.164386:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981e9272c600) 00010000:00010000:1.0:1789762851.164389:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762851.164401:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b04b17 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.164411:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b04b17 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.164419:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b04b17 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.164435:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b04b1e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.164443:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b04b1e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.164450:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b04b1e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.164692:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b04b25 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.164703:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b04b25 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.164709:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b04b25 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789762851.165482:0:5805:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].20, 64 off9472 00000004:00080000:1.0:1789762851.165490:0:5805:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 374469402047ns, next at 375469400530ns, rc = 0 00010000:00010000:1.0:1789762851.165521:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b04b2c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165528:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b04b2c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165534:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b04b2c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165539:0:5805:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b04b2c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165543:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b04b2c lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165549:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b04b2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165569:0:5805:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762851.165582:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b04b25 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165590:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b04b25 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165598:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b04b25 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762851.165603:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762851.165605:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762851.165611:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b04b25 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762851.165613:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b04b2c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165620:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b04b25 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762851.165627:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762851.165632:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff981ec2179500) 00010000:00010000:1.0:1789762851.165637:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762851.165642:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b04b25 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762851.165643:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b04b2c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165660:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5e00/0x416193e012b04b25 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762851.165664:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b04b2c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762851.165684:0:6663:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff981ec2179500) 00010000:00010000:1.0:1789762851.165685:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b04b17 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762851.165687:0:6663:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789762851.165692:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b04b2c lrc: 1/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165695:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b04b17 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762851.165702:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5a00/0x416193e012b04b2c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165705:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b04b17 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165711:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762851.165713:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b04b17 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165734:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b04b17 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762851.165743:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762851.165744:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981e9272c600) 00010000:00010000:2.0:1789762851.165746:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762851.165747:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762851.165749:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762851.165751:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b04b17 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.165757:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4600/0x416193e012b04b17 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762851.165764:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008620e239 reply state 0000000038680f93 transno 4294967559 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b04b1e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762851.165769:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762851.165803:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14329:x1876702015059968:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1487us (1550us total) trans 4294967559 rc 0/0 00000100:00100000:1.0:1789762851.165808:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 306 00010000:00010000:0.0:1789762851.166315:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b04b1e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.166334:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b04b1e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.166339:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b04b1e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.166343:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762851.166345:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b04b1e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.166349:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b04b1e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.166356:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x4009f790 (ffff981ec2179000) 00010000:00010000:0.0:1789762851.166358:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762851.166361:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b04b1e lrc: 2/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.166364:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5400/0x416193e012b04b1e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762851.166717:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762851.166747:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015060224 00000100:00100000:1.0:1789762851.166752:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015060224 00000100:00100000:1.0:1789762851.166775:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 98 00000100:00100000:1.0:1789762851.166781:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:14329:x1876702015060224:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:1.0:1789762851.166856:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14329:x1876702015060224:12345-192.168.203.48@tcp:35:rm.0 Request processed in 75us (140us total) trans 4294967560 rc 0/0 00000100:00100000:1.0:1789762851.166861:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 98 00000100:00100000:3.0:1789762851.167516:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762851.167548:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015060480 00000100:00100000:0.0:1789762851.167553:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015060480 00000100:00100000:0.0:1789762851.167583:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 307 00000100:00100000:0.0:1789762851.167589:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14329:x1876702015060480:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:0.0:1789762851.167612:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1789762851.167625:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184200/0x416193e012b04b09 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6d3 expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.167658:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4000/0x416193e012b0450c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6a9 expref: 13 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.167666:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981e9272c600) 00010000:00010000:0.0:1789762851.167675:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4800/0x416193e012b044aa lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd694 expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.167681:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:0.0:1789762851.167684:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762851.167694:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04b33 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.167699:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04b33 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.167704:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04b33 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.167712:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b3a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.167716:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b3a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.167721:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b3a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.167920:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b04b41 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.167934:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b04b41 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.167940:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b04b41 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762851.168340:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00010000:00010000:0.0:1789762851.168348:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b04b41 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168365:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b04b41 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168370:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b04b41 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168374:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762851.168376:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b04b41 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168380:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b04b41 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168386:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762851.168388:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762851.168390:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b04b41 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168394:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184000/0x416193e012b04b41 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168405:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04b33 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168414:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04b33 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168418:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04b33 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168421:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762851.168423:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04b33 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168426:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04b33 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168431:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3800) 00010000:00010000:0.0:1789762851.168433:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762851.168435:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04b33 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.168438:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185200/0x416193e012b04b33 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762851.168445:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f821c2b4 reply state 000000009146f460 transno 4294967561 ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b3a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762851.168451:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762851.168494:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:14329:x1876702015060480:12345-192.168.203.48@tcp:36:rm.0 Request processed in 903us (982us total) trans 4294967561 rc 0/0 00000100:00100000:0.0:1789762851.168502:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 307 00010000:00010000:1.0:1789762851.168957:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b3a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.168971:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b3a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.168990:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b3a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.168996:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762851.169000:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b3a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.169007:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b3a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.169016:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3d00) 00010000:00010000:1.0:1789762851.169019:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762851.169022:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b3a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762851.169029:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b3a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762851.767160:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1789762852.010595:0:15774:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 375314505627ns, next at 375469400530ns, rc = 0 00000001:00080000:2.0:1789762852.019681:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967561 is committed 00000004:00080000:2.0:1789762852.019692:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 26 committed 00000004:00080000:2.0:1789762852.019704:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 25 committed 00000100:00100000:0.0:1789762852.019812:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022176128:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789762852.019874:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762852.019913:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022176128 00000100:00100000:2.0:1789762852.019922:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022176128 00000100:00080000:3.0:1789762852.019968:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762852.019997:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:2.0:1789762852.020008:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84542d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022176128:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1789762852.020031:0:6647:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:20 count 1 00002000:00010000:2.0:1789762852.020057:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 104857600, mtime: 0x6aad9d19, atime: 0x0, ctime: 0x6aad9d19, blocks: 0x3200b 00010000:00010000:2.0:1789762852.020064:0:6647:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04b48 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762852.020077:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=375s v=(0 0) 00010000:00010000:2.0:1789762852.020081:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762852.020107:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04b48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762852.020120:0:6647:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04b48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762852.020128:0:6647:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04b48 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762852.020143:0:6647:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04b48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762852.020157:0:6647:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762852.020177:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762852.020184:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04b48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762852.020192:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4c400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762852.020196:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04b48 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762852.020204:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04b48 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762852.020219:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04b48 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762852.020224:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04b48 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762852.020236:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762852.020238:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762852.020239:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762852.020562:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84542d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022176128:12345-0@lo:6:osp-syn-0-0.0 Request processed in 556us (686us total) trans 4294968031 rc 0/0 00000100:00100000:2.0:1789762852.020568:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000004:00080000:1.0:1789762852.020595:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981d8387f100/2, rc 0, transno 4294968031 00000100:00100000:1.0:1789762852.020609:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022176128:0@lo:6:osp-syn-0-0.0 00000004:00080000:3.0:1789762852.020809:0:15774:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 375324720978ns, next at 358343111120ns, rc = 0 00000004:00080000:2.0:1789762852.021641:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 21 committed 00010000:00010000:0.0:1789762852.791173:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762853.815232:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762853.986385:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762853.986422:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762853.986447:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762853.986465:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015060608 00000100:00100000:0.0:1789762853.986478:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015060608 00000100:00100000:2.0:1789762853.986482:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015060864 00000100:00100000:2.0:1789762853.986489:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015060864 00000100:00080000:3.0:1789762853.986548:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762853.986558:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 320 00000100:00100000:1.0:1789762853.986559:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015060736 00000100:00100000:2.0:1789762853.986569:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 145 00000100:00100000:1.0:1789762853.986574:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015060736 00000100:00100000:0.0:1789762853.986575:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015060608:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762853.986588:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015060864:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:3.0:1789762853.986647:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762853.986649:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015060864:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 64us (222us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762853.986652:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015060608:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 80us (319us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762853.986659:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 145 00000100:00100000:0.0:1789762853.986674:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 320 00000100:00100000:1.0:1789762853.986702:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 288 00000100:00100000:1.0:1789762853.986726:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015060736:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762853.986816:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015060736:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 102us (368us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762853.986822:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 288 00000100:00100000:3.0:1789762853.989834:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762853.989868:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762853.989898:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015060992 00000100:00100000:1.0:1789762853.989905:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015060992 00000100:00100000:0.0:1789762853.989926:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015061120 00000100:00100000:0.0:1789762853.989933:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015061120 00000100:00080000:3.0:1789762853.989954:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789762853.989970:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762853.989987:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 184 00000100:00100000:0.0:1789762853.990001:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 289 00000100:00100000:1.0:1789762853.990008:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2362:x1876702015060992:12345-192.168.203.48@tcp:9:kworker.0 00000100:00100000:0.0:1789762853.990015:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015061120:12345-192.168.203.48@tcp:400:kworker.0 00000020:00000080:1.0:1789762853.990036:0:6648:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x416193e012b0435a 00000100:00080000:1.0:1789762853.990052:0:6648:0:(import.c:65:import_set_state_nolock()) ffff981ebfa2f800 : changing import state from FULL to CLOSED 00000100:00100000:0.0:1789762853.990066:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015061120:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 51us (196us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762853.990076:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 289 00010000:00010000:2.0:1789762853.990117:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00000004:1.0:1789762853.990142:0:6648:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967416 00000100:00100000:1.0:1789762853.990396:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+2:2362:x1876702015060992:12345-192.168.203.48@tcp:9:kworker.0 Request processed in 398us (564us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762853.990403:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 184 00000100:00100000:0.0:1789762854.007246:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789762854.007268:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022176256:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762854.007316:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762854.007353:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022176256 00000100:00100000:2.0:1789762854.007359:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022176256 00000100:00100000:2.0:1789762854.007418:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 121 00000100:00100000:2.0:1789762854.007430:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84542680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022176256:12345-0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1789762854.007435:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00000024:2.0:1789762854.007462:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762854.007503:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84542680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022176256:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (188us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762854.007509:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 121 00000100:00100000:0.0:1789762854.007549:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022176256:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762854.519271:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead4f4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022176384:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762854.519277:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022176512:0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.519295:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022176640:0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.519364:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022176768:0@lo:400:kworker.0 00000100:00100000:3.0:1789762854.519389:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762854.519403:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762854.519404:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762854.519465:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022176640 00000100:00100000:1.0:1789762854.519475:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022176640 00000100:00100000:3.0:1789762854.519505:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022176896:0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.519515:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022177024:0@lo:400:kworker.0 00000100:00100000:3.0:1789762854.519560:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762854.519566:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:2.0:1789762854.519579:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762854.519581:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022176512 00000100:00100000:1.0:1789762854.519587:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022176640:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.519589:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022176512 00000100:00100000:3.0:1789762854.519641:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022176896 00000100:00100000:3.0:1789762854.519648:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022176896 00000100:00100000:0.0:1789762854.519654:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 321 00000100:00100000:2.0:1789762854.519670:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022177024 00000100:00100000:0.0:1789762854.519672:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022176512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762854.519678:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022177024 00000100:00100000:1.0:1789762854.519683:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022176640:12345-0@lo:400:kworker.0 Request processed in 100us (283us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762854.519695:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00100000:3.0:1789762854.519712:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 291 00000100:00100000:3.0:1789762854.519729:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022176896:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762854.519751:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022176512:12345-0@lo:400:kworker.0 Request processed in 83us (354us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762854.519754:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:0.0:1789762854.519774:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 321 00000100:00100000:2.0:1789762854.519780:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022177024:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762854.519791:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022176896:12345-0@lo:400:kworker.0 Request processed in 68us (238us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762854.519792:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022176384 00000100:00100000:1.0:1789762854.519803:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022176384 00000100:00100000:3.0:1789762854.519805:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 291 00000100:00100000:2.0:1789762854.519855:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022177024:12345-0@lo:400:kworker.0 Request processed in 81us (285us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762854.519856:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022176512:0@lo:400:kworker.0 00000100:00080000:3.0:1789762854.519865:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762854.519875:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00000100:00080000:3.0:1789762854.519877:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762854.519882:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00100000:1.0:1789762854.519904:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022176384:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762854.519925:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022177024:0@lo:400:kworker.0 00000100:00100000:3.0:1789762854.519933:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022176896:0@lo:400:kworker.0 00002000:00000024:1.0:1789762854.519967:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762854.519992:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022176640:0@lo:400:kworker.0 00000100:00100000:3.0:1789762854.520011:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762854.520043:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022176384:12345-0@lo:13:osp-pre-1-0.0 Request processed in 150us (661us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762854.520052:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000100:00100000:2.0:1789762854.520070:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022176768 00000100:00100000:2.0:1789762854.520076:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022176768 00000100:00080000:3.0:1789762854.520154:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762854.520165:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:2.0:1789762854.520179:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022176768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762854.520195:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022176384:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762854.520223:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022176768:12345-0@lo:400:kworker.0 Request processed in 50us (216us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762854.520230:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:3.0:1789762854.520266:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022176768:0@lo:400:kworker.0 00010000:00010000:0.0:1789762854.839188:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:1.0:1789762855.551174:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:1.0:1789762855.551181:0:7605:0:(genops.c:885:class_export_put()) final put ffff981ebfa2e800/f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:2.0:1789762855.551206:0:37:0:(genops.c:844:class_export_destroy()) destroying export ffff981ebfa2e800/f1bd28de-ee78-4d18-bad3-fa7cec968893 for lustre-OST0001 00000020:00000080:2.0:1789762855.551253:0:37:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff981ebfa2f800 for lustre-OST0001 00010000:00010000:0.0:1789762855.863176:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762856.887261:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789762857.097358:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968031 is committed 00010000:00010000:2.0:1789762857.911209:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762858.935201:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762859.106298:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762859.106398:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015061248 00000100:00100000:3.0:1789762859.106407:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762859.106415:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015061248 00000100:00100000:3.0:1789762859.106453:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762859.106458:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015061376 00000100:00100000:1.0:1789762859.106466:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015061376 00000100:00100000:0.0:1789762859.106488:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 322 00000100:00100000:3.0:1789762859.106505:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762859.106511:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015061248:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762859.106553:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015061632 00000100:00100000:3.0:1789762859.106555:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015061504 00000100:00100000:1.0:1789762859.106558:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015061632 00000100:00100000:3.0:1789762859.106564:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015061504 00000100:00100000:0.0:1789762859.106603:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015061248:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 98us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762859.106614:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 322 00000100:00100000:1.0:1789762859.106624:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 293 00000100:00100000:1.0:1789762859.106646:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015061376:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762859.106651:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 147 00000100:00100000:3.0:1789762859.106673:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015061504:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762859.106680:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 294 00000100:00100000:0.0:1789762859.106690:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015061632:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762859.106710:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015061376:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 71us (305us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762859.106718:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762859.106723:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 293 00000100:00080000:2.0:1789762859.106726:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762859.106749:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015061632:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 50us (236us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762859.106751:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015061504:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 70us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762859.106770:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 294 00000100:00100000:3.0:1789762859.106786:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 147 00000100:00100000:0.0:1789762859.128216:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022177152:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762859.128276:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762859.128311:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022177152 00000100:00100000:2.0:1789762859.128323:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022177152 00000100:00080000:1.0:1789762859.128394:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762859.128409:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 123 00000100:00100000:2.0:1789762859.128419:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022177152:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762859.128454:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762859.128510:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022177152:12345-0@lo:13:osp-pre-0-0.0 Request processed in 91us (231us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762859.128519:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 123 00000100:00100000:0.0:1789762859.128538:0:3271:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294968031 gen 2 00000004:00080000:0.0:1789762859.128545:0:3271:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981d8387f100, transno 4294968031 00000100:00100000:0.0:1789762859.128571:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022177152:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762859.639257:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789762859.639257:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789762859.639283:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb15500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022177408:0@lo:400:kworker.0 00000100:00100000:1.0:1789762859.639284:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead4f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022177280:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762859.639312:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e879c7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022177792:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.639344:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e879c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022177920:0@lo:400:kworker.0 00000100:00100000:0.0:1789762859.639367:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762859.639372:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762859.639425:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762859.639431:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762859.639451:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e879c5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022177536:0@lo:400:kworker.0 00000100:00100000:1.0:1789762859.639451:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e879c4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022177664:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.639484:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022177408 00000100:00100000:2.0:1789762859.639491:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022177408 00000100:00100000:1.0:1789762859.639505:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762859.639532:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022177280 00000100:00100000:0.0:1789762859.639539:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022177280 00000100:00100000:3.0:1789762859.639547:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022177792 00000100:00100000:2.0:1789762859.639553:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 323 00000100:00100000:3.0:1789762859.639555:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022177792 00000100:00100000:2.0:1789762859.639565:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850de450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022177408:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762859.639573:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022177920 00000100:00100000:1.0:1789762859.639579:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022177920 00000100:00100000:0.0:1789762859.639601:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 124 00000100:00100000:0.0:1789762859.639611:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022177280:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762859.639627:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850de450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022177408:12345-0@lo:400:kworker.0 Request processed in 63us (262us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762859.639634:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:2.0:1789762859.639639:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 323 00002000:00000024:0.0:1789762859.639644:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762859.639644:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022177664 00000100:00100000:1.0:1789762859.639651:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022177920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.639657:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022177664 00000100:00100000:2.0:1789762859.639683:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 295 00000100:00100000:2.0:1789762859.639697:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022177792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.639723:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:1.0:1789762859.639733:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022177920:12345-0@lo:400:kworker.0 Request processed in 81us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762859.639734:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022177280:12345-0@lo:13:osp-pre-1-0.0 Request processed in 117us (360us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762859.639749:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022177664:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.639751:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022177792:12345-0@lo:400:kworker.0 Request processed in 55us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762859.639763:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00100000:0.0:1789762859.639766:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 124 00000100:00100000:2.0:1789762859.639782:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 295 00000100:00080000:0.0:1789762859.639809:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762859.639815:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762859.639826:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e879c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022177920:0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.639860:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022177664:12345-0@lo:400:kworker.0 Request processed in 118us (352us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762859.639864:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022177280:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762859.639881:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762859.639896:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:0.0:1789762859.639906:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e879c4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022177664:0@lo:400:kworker.0 00000100:00100000:2.0:1789762859.639937:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e879c7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022177792:0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.639957:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022177536 00000100:00100000:1.0:1789762859.639958:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb15500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022177408:0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.639969:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022177536 00000100:00100000:3.0:1789762859.640002:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:3.0:1789762859.640009:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022177536:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762859.640051:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022177536:12345-0@lo:400:kworker.0 Request processed in 44us (179us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762859.640056:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:1.0:1789762859.640121:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e879c5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022177536:0@lo:400:kworker.0 00010000:00010000:0.0:1789762859.959194:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762860.983291:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762862.007209:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762863.031180:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762864.055193:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762864.226286:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762864.226367:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762864.226394:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762864.226399:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015061760 00000100:00100000:0.0:1789762864.226408:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015061760 00000100:00100000:2.0:1789762864.226418:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015062016 00000100:00100000:3.0:1789762864.226424:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762864.226426:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015062016 00000100:00100000:0.0:1789762864.226470:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 324 00000100:00080000:3.0:1789762864.226477:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762864.226486:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015061760:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762864.226487:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015061888 00000100:00100000:1.0:1789762864.226495:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015061888 00000100:00100000:3.0:1789762864.226500:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015062144 00000100:00100000:3.0:1789762864.226506:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015062144 00000100:00100000:2.0:1789762864.226508:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 149 00000100:00100000:2.0:1789762864.226526:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015062016:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762864.226549:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 298 00000100:00100000:0.0:1789762864.226561:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015061760:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 79us (280us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762864.226562:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015061888:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762864.226570:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 299 00000100:00100000:0.0:1789762864.226581:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 324 00000100:00100000:3.0:1789762864.226588:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015062144:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762864.226598:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015062016:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 75us (227us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762864.226608:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 149 00000100:00100000:1.0:1789762864.226612:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015061888:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 53us (216us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762864.226620:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 298 00000100:00100000:3.0:1789762864.226638:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015062144:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 53us (211us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762864.226647:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 299 00000100:00080000:1.0:1789762864.226649:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762864.247265:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022178048:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762864.247313:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762864.247345:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022178048 00000100:00100000:2.0:1789762864.247353:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022178048 00000100:00080000:1.0:1789762864.247384:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762864.247395:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 125 00000100:00100000:2.0:1789762864.247404:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022178048:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762864.247431:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762864.247473:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022178048:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (159us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762864.247478:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 125 00000100:00100000:0.0:1789762864.247524:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022178048:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762864.759238:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022178304:0@lo:400:kworker.0 00000100:00100000:3.0:1789762864.759262:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb15500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022178432:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.759271:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac5a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022178560:0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.759296:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022178688:0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.759313:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762864.759324:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762864.759340:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022178304 00000100:00100000:1.0:1789762864.759340:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762864.759349:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022178304 00000100:00100000:2.0:1789762864.759371:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead4f7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022178176:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762864.759380:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e902dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022178944:0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.759394:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762864.759405:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 325 00000100:00100000:0.0:1789762864.759417:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022178304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.759432:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762864.759436:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022178432 00000100:00100000:3.0:1789762864.759437:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762864.759442:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022178432 00000100:00100000:2.0:1789762864.759471:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc554380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022178816:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.759477:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022178304:12345-0@lo:400:kworker.0 Request processed in 59us (161us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762864.759478:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022178560 00000100:00100000:1.0:1789762864.759483:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022178560 00000100:00100000:0.0:1789762864.759491:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 325 00000100:00100000:3.0:1789762864.759494:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 300 00000100:00100000:2.0:1789762864.759503:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762864.759509:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022178432:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.759525:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022178816 00000100:00100000:1.0:1789762864.759529:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022178816 00000100:00100000:0.0:1789762864.759536:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022178688 00000100:00100000:0.0:1789762864.759542:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022178688 00000100:00100000:2.0:1789762864.759549:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851b9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022178304:0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.759558:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 301 00000100:00100000:3.0:1789762864.759566:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022178432:12345-0@lo:400:kworker.0 Request processed in 62us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762864.759569:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022178560:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.759582:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022178944 00000100:00100000:3.0:1789762864.759582:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 300 00000100:00100000:0.0:1789762864.759587:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022178944 00000100:00100000:2.0:1789762864.759597:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:2.0:1789762864.759607:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022178816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.759620:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022178560:12345-0@lo:400:kworker.0 Request processed in 52us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762864.759621:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:1.0:1789762864.759631:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 301 00000100:00100000:3.0:1789762864.759633:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb15500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022178432:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.759636:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022178688:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.759654:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022178816:12345-0@lo:400:kworker.0 Request processed in 48us (150us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762864.759668:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00100000:1.0:1789762864.759669:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:1.0:1789762864.759681:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022178944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762864.759691:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc554380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022178816:0@lo:400:kworker.0 00000100:00100000:0.0:1789762864.759696:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022178688:12345-0@lo:400:kworker.0 Request processed in 65us (358us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762864.759704:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00080000:2.0:1789762864.759714:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762864.759721:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022178944:12345-0@lo:400:kworker.0 Request processed in 44us (292us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762864.759728:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00080000:2.0:1789762864.759732:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762864.759743:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022178176 00000100:00100000:3.0:1789762864.759748:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022178176 00000100:00100000:0.0:1789762864.759757:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac5a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022178560:0@lo:400:kworker.0 00000100:00100000:1.0:1789762864.759769:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e902dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022178944:0@lo:400:kworker.0 00000100:00100000:2.0:1789762864.759791:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022178688:0@lo:400:kworker.0 00000100:00100000:3.0:1789762864.759792:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 126 00000100:00100000:3.0:1789762864.759807:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022178176:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789762864.759834:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762864.759894:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022178176:12345-0@lo:13:osp-pre-1-0.0 Request processed in 74us (450us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762864.759900:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 126 00000100:00100000:2.0:1789762864.759916:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022178176:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789762865.079235:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762866.103235:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762867.127176:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762868.151187:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762869.175302:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762869.346311:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762869.346344:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762869.346369:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762869.346373:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015062272 00000100:00100000:0.0:1789762869.346385:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015062272 00000100:00100000:3.0:1789762869.346385:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015062400 00000100:00100000:3.0:1789762869.346392:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015062400 00000100:00100000:2.0:1789762869.346398:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:2.0:1789762869.346424:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762869.346446:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 326 00000100:00100000:2.0:1789762869.346448:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015062656 00000100:00100000:2.0:1789762869.346454:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015062656 00000100:00100000:3.0:1789762869.346457:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 303 00000100:00100000:0.0:1789762869.346460:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850de050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015062272:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762869.346465:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015062528 00000100:00100000:3.0:1789762869.346471:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015062400:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762869.346472:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015062528 00000100:00100000:2.0:1789762869.346503:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 304 00000100:00100000:2.0:1789762869.346512:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015062656:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762869.346526:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 152 00000100:00100000:1.0:1789762869.346540:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015062528:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762869.346551:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850de050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015062272:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 92us (242us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762869.346553:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015062656:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 41us (152us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762869.346553:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015062400:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 80us (200us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762869.346573:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 303 00000100:00100000:0.0:1789762869.346580:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 326 00000100:00100000:2.0:1789762869.346585:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 304 00000100:00100000:1.0:1789762869.346587:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015062528:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 46us (212us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762869.346599:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 152 00000100:00080000:2.0:1789762869.346641:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762869.346647:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762869.367358:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e902dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022179072:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762869.367423:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762869.367456:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022179072 00000100:00100000:2.0:1789762869.367465:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022179072 00000100:00080000:3.0:1789762869.367506:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762869.367544:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:2.0:1789762869.367555:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022179072:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762869.367589:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762869.367673:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022179072:12345-0@lo:13:osp-pre-0-0.0 Request processed in 118us (246us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762869.367707:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00000100:00100000:0.0:1789762869.367750:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e902dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022179072:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762869.879298:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762869.879298:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8aa7aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022179456:0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.879314:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e879c4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022179200:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762869.879317:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8aa79f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022179584:0@lo:400:kworker.0 00000100:00100000:1.0:1789762869.879353:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8aa79880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022179712:0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.879384:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762869.879384:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762869.879397:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762869.879426:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022179456 00000100:00100000:0.0:1789762869.879434:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022179456 00000100:00100000:3.0:1789762869.879451:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8aa78380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022179840:0@lo:400:kworker.0 00000100:00100000:2.0:1789762869.879456:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8aa7b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022179968:0@lo:400:kworker.0 00000100:00100000:1.0:1789762869.879479:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022179712 00000100:00100000:1.0:1789762869.879484:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022179712 00000100:00100000:3.0:1789762869.879500:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762869.879507:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 305 00000100:00100000:2.0:1789762869.879517:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762869.879522:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022179456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762869.879525:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:1.0:1789762869.879539:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022179712:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.879543:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022179840 00000100:00100000:3.0:1789762869.879548:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022179840 00000100:00100000:2.0:1789762869.879585:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022179968 00000100:00100000:2.0:1789762869.879594:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022179968 00000100:00100000:0.0:1789762869.879604:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022179456:12345-0@lo:400:kworker.0 Request processed in 87us (223us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762869.879609:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:1.0:1789762869.879610:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022179712:12345-0@lo:400:kworker.0 Request processed in 70us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762869.879622:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 305 00000100:00100000:3.0:1789762869.879626:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022179840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762869.879632:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1789762869.879657:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1789762869.879659:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022179200 00000100:00100000:0.0:1789762869.879664:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022179200 00000100:00100000:3.0:1789762869.879672:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022179840:12345-0@lo:400:kworker.0 Request processed in 50us (171us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762869.879678:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022179968:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.879685:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:0.0:1789762869.879697:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 128 00000100:00100000:1.0:1789762869.879703:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8aa79880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022179712:0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.879709:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022179200:12345-0@lo:13:osp-pre-1-0.0 00000100:00080000:3.0:1789762869.879710:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789762869.879717:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762869.879740:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022179968:12345-0@lo:400:kworker.0 Request processed in 69us (224us total) trans 0 rc 0/0 00002000:00000024:0.0:1789762869.879749:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762869.879758:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:3.0:1789762869.879759:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8aa78380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022179840:0@lo:400:kworker.0 00000100:00100000:1.0:1789762869.879763:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762869.879799:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e879c4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022179200:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762869.879800:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022179200:12345-0@lo:13:osp-pre-1-0.0 Request processed in 95us (418us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762869.879804:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e902dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022179328:0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.879815:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 128 00000100:00100000:3.0:1789762869.879853:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022179584 00000100:00100000:2.0:1789762869.879857:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8aa7b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022179968:0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.879859:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022179584 00000100:00100000:1.0:1789762869.879865:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762869.879879:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8aa7aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022179456:0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.879900:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022179328 00000100:00100000:0.0:1789762869.879905:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022179328 00000100:00100000:3.0:1789762869.879915:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00080000:2.0:1789762869.879922:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762869.879927:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa79880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022179584:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.879949:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 327 00000100:00100000:0.0:1789762869.879957:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022179328:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762869.879968:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa79880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022179584:12345-0@lo:400:kworker.0 Request processed in 44us (210us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762869.879982:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00000100:00100000:1.0:1789762869.879995:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8aa79f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022179584:0@lo:400:kworker.0 00000100:00100000:1.0:1789762869.880017:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e902dc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022179328:0@lo:400:kworker.0 00000100:00100000:0.0:1789762869.880022:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850df050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022179328:12345-0@lo:400:kworker.0 Request processed in 65us (159us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762869.880028:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 327 00010000:00010000:2.0:1789762870.199282:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762871.223154:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762871.874392:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762871.874434:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015062784 00000100:00100000:2.0:1789762871.874444:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015062784 00000100:00080000:0.0:1789762871.874477:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762871.874509:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 308 00000100:00100000:2.0:1789762871.874520:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:14823:x1876702015062784:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:2.0:1789762871.874538:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762871.874557:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ca00/0x416193e012b04b4f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6da expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.874621:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b04b56 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.874639:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b04b56 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.874649:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b04b56 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762871.874678:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762871.874681:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762871.874708:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b04b56 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.874728:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ca00/0x416193e012b04b4f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6da expref: 7 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.874740:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b04b56 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6da expref: 6 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.874746:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26500) 00010000:00010000:2.0:1789762871.874748:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762871.874751:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4d200, rc 0) 00000100:00100000:2.0:1789762871.874803:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14823:x1876702015062784:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 286us (416us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762871.874808:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 308 00000100:00100000:0.0:1789762871.876237:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762871.876272:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015062912 00000100:00100000:1.0:1789762871.876279:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015062912 00000100:00100000:1.0:1789762871.876311:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 309 00000100:00100000:1.0:1789762871.876319:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015062912:12345-192.168.203.48@tcp:41:mkdir.0 00000004:00000024:1.0:1789762871.876345:0:5805:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:1.0:1789762871.876386:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015062912:12345-192.168.203.48@tcp:41:mkdir.0 Request processed in 67us (150us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762871.876393:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 309 00000100:00100000:0.0:1789762871.876412:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762871.876442:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015063168 00000100:00100000:1.0:1789762871.876446:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015063168 00000100:00100000:1.0:1789762871.876466:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 310 00000100:00100000:1.0:1789762871.876472:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14823:x1876702015063168:12345-192.168.203.48@tcp:36:mkdir.0 00010000:00010000:1.0:1789762871.876497:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762871.876510:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b04b56 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6da expref: 6 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.876522:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26500) 00010000:00010000:1.0:1789762871.876529:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762871.876729:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b5d lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.876737:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b5d lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.876744:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b5d lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.876756:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04b64 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.876761:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04b64 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.876766:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04b64 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762871.877898:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00010000:00010000:1.0:1789762871.877906:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b5d lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.877912:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b5d lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.877916:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b5d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.877921:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.877924:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b5d lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.877943:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b5d lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.877951:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179800) 00010000:00010000:1.0:1789762871.877953:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762871.877956:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b5d lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.877960:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b5d lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762871.877972:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000068fd4186 reply state 00000000b6bef2a8 transno 4294967562 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04b64 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.877978:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762871.878020:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:14823:x1876702015063168:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 1548us (1612us total) trans 4294967562 rc 0/0 00000100:00100000:1.0:1789762871.878027:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 310 00010000:00010000:1.0:1789762871.878694:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04b64 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.878703:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04b64 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.878708:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04b64 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.878713:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.878715:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04b64 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.878724:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04b64 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.878732:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179500) 00010000:00010000:1.0:1789762871.878734:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762871.878737:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04b64 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.878741:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04b64 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762871.884699:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762871.884766:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015063296 00000100:00100000:0.0:1789762871.884771:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015063296 00000100:00100000:0.0:1789762871.884803:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 311 00000100:00100000:0.0:1789762871.884812:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14824:x1876702015063296:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762871.884838:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762871.884850:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b6b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6e1 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.884876:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04b72 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.884883:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04b72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.884890:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04b72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762871.884903:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762871.884906:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762871.884925:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04b72 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.884948:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185c00/0x416193e012b04b6b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6e1 expref: 8 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.884958:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04b72 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6e1 expref: 7 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.884963:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3a00) 00010000:00010000:0.0:1789762871.884964:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762871.884966:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86184a00, rc 0) 00000100:00100000:0.0:1789762871.884999:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:14824:x1876702015063296:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 190us (302us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762871.885004:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 311 00000100:00100000:0.0:1789762871.886509:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762871.886582:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015063552 00000100:00100000:1.0:1789762871.886588:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015063552 00000100:00100000:1.0:1789762871.886635:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 312 00000100:00100000:1.0:1789762871.886644:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:14824:x1876702015063552:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762871.886653:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762871.886654:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1789762871.886674:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86184a00/0x416193e012b04b72 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6e1 expref: 7 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886684:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3a00) 00010000:00010000:1.0:1789762871.886689:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762871.886696:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04b79 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6e8 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886750:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b04b80 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886757:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b04b80 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886766:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b04b80 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886781:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04b87 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886791:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04b87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886800:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04b87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886886:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b8e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886893:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886899:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762871.886905:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00010000:00010000:1.0:1789762871.886912:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886917:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b8e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886932:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b8e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886936:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.886939:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b8e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886943:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b8e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886949:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981ec2179100) 00010000:00010000:1.0:1789762871.886951:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762871.886953:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b8e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886957:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04b8e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886968:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b04b80 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886973:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b04b80 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886977:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b04b80 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886981:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.886982:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b04b80 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886986:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b04b80 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886991:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179800) 00010000:00010000:1.0:1789762871.886992:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762871.886995:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b04b80 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.886999:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5800/0x416193e012b04b80 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.887005:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04b87 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.887009:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04b87 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.887013:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04b87 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.887018:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.887020:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04b87 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.887024:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04b87 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.887029:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179000) 00010000:00010000:1.0:1789762871.887031:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762871.887032:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04b87 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.887036:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4c00/0x416193e012b04b87 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.887052:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04b79 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6e8 expref: 7 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.887062:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179800) 00010000:00010000:1.0:1789762871.887064:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762871.887067:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04b79 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6e8 expref: 7 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.887075:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb4a00, rc 0) 00000100:00100000:1.0:1789762871.887134:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14824:x1876702015063552:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 491us (625us total) trans 0 rc 301/301 00000100:00100000:1.0:1789762871.887142:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 312 00000100:00100000:1.0:1789762871.888052:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762871.888083:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015063680 00000100:00100000:3.0:1789762871.888103:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015063680 00000100:00100000:3.0:1789762871.888135:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 313 00000100:00100000:3.0:1789762871.888142:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:14824:x1876702015063680:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762871.888150:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762871.888160:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3000/0x416193e012b04b95 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6ef expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.888190:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b04b9c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.888198:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b04b9c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.888204:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b04b9c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762871.888219:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762871.888227:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762871.888236:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b04b9c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.888248:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3000/0x416193e012b04b95 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6ef expref: 8 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.888258:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b04b9c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6ef expref: 7 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.888264:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c500) 00010000:00010000:3.0:1789762871.888266:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762871.888268:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ec21d3200, rc 0) 00000100:00100000:3.0:1789762871.888296:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:14824:x1876702015063680:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 155us (246us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762871.888301:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 313 00000100:00100000:0.0:1789762871.890395:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762871.890460:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015063936 00000100:00100000:1.0:1789762871.890466:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015063936 00000100:00100000:1.0:1789762871.890500:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 314 00000100:00100000:1.0:1789762871.890509:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:14824:x1876702015063936:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762871.890517:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762871.890526:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04ba3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6f6 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890555:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04baa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890563:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04baa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890569:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04baa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890583:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bb1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890588:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bb1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890592:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bb1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762871.890642:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00010000:00010000:1.0:1789762871.890678:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04baa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890683:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04baa lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890687:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04baa lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890691:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.890693:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04baa lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890697:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04baa lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890704:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c500) 00010000:00010000:1.0:1789762871.890706:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762871.890709:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04baa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890717:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04baa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890724:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bb1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890728:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bb1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890733:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bb1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890736:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.890738:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bb1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890741:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bb1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890746:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179500) 00010000:00010000:1.0:1789762871.890747:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762871.890749:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bb1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890752:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bb1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890776:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04ba3 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6f6 expref: 9 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890791:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c500) 00010000:00010000:1.0:1789762871.890794:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789762871.890798:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04ba3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6f6 expref: 9 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.890809:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb5200, rc 0) 00000100:00100000:1.0:1789762871.890858:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:14824:x1876702015063936:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 351us (465us total) trans 4294967563 rc 301/301 00000100:00100000:1.0:1789762871.890864:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 314 00000100:00100000:0.0:1789762871.891977:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762871.892014:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015064192 00000100:00100000:1.0:1789762871.892018:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015064192 00000100:00100000:1.0:1789762871.892043:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 315 00000100:00100000:1.0:1789762871.892049:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:14824:x1876702015064192:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:1.0:1789762871.892079:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04bbf lrc: 3/0,1 mode: --/PW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.892105:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04bbf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.892112:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04bbf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762871.892207:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000091180724 reply state 00000000aa8c5d27 transno 4294967564 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04bbf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.892225:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:1.0:1789762871.892228:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00000100:00100000:1.0:1789762871.892267:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14824:x1876702015064192:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 218us (292us total) trans 4294967564 rc 0/0 00000100:00100000:1.0:1789762871.892274:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 315 00010000:00010000:1.0:1789762871.892776:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04bbf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.892788:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04bbf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.892797:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04bbf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.892804:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.892809:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04bbf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.892818:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04bbf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.892841:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981ec2179d00) 00010000:00010000:1.0:1789762871.892844:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789762871.892847:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04bbf lrc: 2/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.892853:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04bbf lrc: 0/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762871.892881:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762871.892913:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015064448 00000100:00100000:1.0:1789762871.892918:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015064448 00000100:00100000:1.0:1789762871.892957:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 99 00000100:00100000:1.0:1789762871.892966:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:14824:x1876702015064448:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:1.0:1789762871.893069:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14824:x1876702015064448:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 105us (189us total) trans 4294967565 rc 0/0 00000100:00100000:1.0:1789762871.893076:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 99 00000100:00100000:1.0:1789762871.897746:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762871.897772:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015064576 00000100:00100000:1.0:1789762871.897778:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015064576 00000100:00100000:1.0:1789762871.897818:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 316 00000100:00100000:1.0:1789762871.897843:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14825:x1876702015064576:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:1.0:1789762871.897853:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762871.897865:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04bc6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd6fd expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.897904:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bcd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.897911:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bcd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.897917:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bcd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.897927:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04bd4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.897932:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04bd4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.897938:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04bd4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.897982:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04bdb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.897988:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04bdb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.897993:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04bdb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762871.898006:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00000004:00200000:1.0:1789762871.898008:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762871.898017:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04bdb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898023:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bcd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898028:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bcd lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898033:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bcd lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898038:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.898040:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bcd lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898045:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bcd lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898053:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c500) 00010000:00010000:1.0:1789762871.898056:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762871.898059:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bcd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898063:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04bcd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898072:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04bd4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898076:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04bd4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898080:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04bd4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898109:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.898113:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04bd4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898118:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04bd4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898126:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179500) 00010000:00010000:1.0:1789762871.898128:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762871.898131:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04bd4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898135:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04bd4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898166:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5200/0x416193e012b04bc6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6fd expref: 12 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898180:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04bdb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6fd expref: 11 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.898185:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981ec2179800) 00010000:00010000:1.0:1789762871.898187:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762871.898189:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb4a00, rc 0) 00000100:00100000:1.0:1789762871.898220:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14825:x1876702015064576:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 380us (475us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762871.898227:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 316 00000100:00100000:1.0:1789762871.899513:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762871.899555:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015064832 00000100:00100000:0.0:1789762871.899561:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015064832 00000100:00100000:0.0:1789762871.899598:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 317 00000100:00100000:0.0:1789762871.899618:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14825:x1876702015064832:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:0.0:1789762871.899639:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762871.899653:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4a00/0x416193e012b04bdb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6fd expref: 11 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.899667:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981ec2179800) 00010000:00010000:0.0:1789762871.899678:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762871.899699:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b04be2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.899707:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b04be2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762871.899714:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b04be2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762871.899862:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001720f5b6 reply state 0000000018589937 transno 4294967566 ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b04be2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762871.899880:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762871.899885:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00000100:00100000:0.0:1789762871.899941:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14825:x1876702015064832:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 331us (426us total) trans 4294967566 rc 0/0 00000100:00100000:0.0:1789762871.899950:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 317 00010000:00010000:1.0:1789762871.900501:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b04be2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.900508:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b04be2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.900514:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b04be2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.900519:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.900522:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b04be2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.900527:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b04be2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.900536:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:1.0:1789762871.900538:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789762871.900541:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b04be2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.900546:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86185400/0x416193e012b04be2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.921118:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762871.921197:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015064960 00000100:00100000:3.0:1789762871.921205:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015064960 00000100:00100000:3.0:1789762871.921256:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 318 00000100:00100000:3.0:1789762871.921265:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa79c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14831:x1876702015064960:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762871.921281:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762871.921304:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3000/0x416193e012b04be9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd704 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921345:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b04bf0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921354:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b04bf0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921374:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b04bf0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921384:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b04bf7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921388:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b04bf7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921393:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b04bf7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921451:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3800/0x416193e012b04bfe lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921457:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3800/0x416193e012b04bfe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921462:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3800/0x416193e012b04bfe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762871.921483:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00000004:00200000:3.0:1789762871.921485:0:5805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762871.921497:0:5805:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3800/0x416193e012b04bfe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921504:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b04bf0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921509:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b04bf0 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921513:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b04bf0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921517:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762871.921520:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b04bf0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921525:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b04bf0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921532:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c500) 00010000:00010000:3.0:1789762871.921534:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762871.921536:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b04bf0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921540:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b04bf0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921547:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b04bf7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921555:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b04bf7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921559:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b04bf7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921562:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762871.921563:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b04bf7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921567:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b04bf7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921571:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272ce00) 00010000:00010000:3.0:1789762871.921572:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762871.921574:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b04bf7 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921578:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b04bf7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921595:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3000/0x416193e012b04be9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd704 expref: 13 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921603:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3800/0x416193e012b04bfe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd704 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.921608:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:3.0:1789762871.921609:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789762871.921611:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ec21d3800, rc 0) 00000100:00100000:3.0:1789762871.921649:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa79c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14831:x1876702015064960:12345-192.168.203.48@tcp:101:dd.0 Request processed in 387us (537us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762871.921654:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 318 00000100:00100000:0.0:1789762871.922753:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762871.922789:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015065216 00000100:00100000:1.0:1789762871.922794:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015065216 00000100:00100000:1.0:1789762871.922834:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 319 00000100:00100000:1.0:1789762871.922843:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14831:x1876702015065216:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762871.922852:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762871.922860:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04c05 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd70b expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.922928:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04c0c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.922945:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04c0c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.922952:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04c0c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.922995:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04c13 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.923002:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04c13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.923006:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04c13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789762871.923464:0:6611:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x15:0x0], index 0: 0 00010000:00010000:1.0:1789762871.924728:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04c1a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.924736:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04c1a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.924745:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04c1a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.924791:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04c0c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.924798:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04c0c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.924802:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04c0c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.924807:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762871.924809:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04c0c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.924813:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04c0c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.924819:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:1.0:1789762871.924834:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762871.924836:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04c0c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.924844:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5600/0x416193e012b04c0c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762871.924852:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000064b76a67 reply state 0000000008a7759f transno 4294967567 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04c13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762871.924857:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789762871.924887:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4e00/0x416193e012b04c05 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd70b expref: 15 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.924895:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04c1a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd70b expref: 14 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.924901:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff981ec2179400) 00010000:00010000:1.0:1789762871.924902:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789762871.924904:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfeb5000, rc 0) 00000100:00100000:1.0:1789762871.924935:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:14831:x1876702015065216:12345-192.168.203.48@tcp:101:dd.0 Request processed in 2094us (2184us total) trans 4294967567 rc 0/0 00000100:00100000:1.0:1789762871.924940:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 319 00010000:00010000:2.0:1789762871.925662:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04c13 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.925671:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04c13 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.925686:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04c13 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.925690:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762871.925692:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04c13 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.925696:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04c13 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.925704:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x4009f790 (ffff981ec2179500) 00010000:00010000:2.0:1789762871.925706:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762871.925708:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04c13 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.925712:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb4200/0x416193e012b04c13 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762871.928389:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762871.928423:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015065344 00000100:00100000:2.0:1789762871.928428:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015065344 00000100:00080000:0.0:1789762871.928470:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762871.928475:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 354351174380 time 354351252788 state 0x0 ref 1 00000100:00100000:2.0:1789762871.928487:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 354401252788 time 354351252788 state 0x1 ref 1 00000100:00100000:2.0:1789762871.928504:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 2 deadline 354401252788 time 395232413271 state 0x0 ref 1 00000100:00100000:2.0:1789762871.928513:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015065344:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:2.0:1789762871.928549:0:14913:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789762871.928554:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04c28 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.928564:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:2.0:1789762871.928568:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762871.928572:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04c28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762871.928581:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04c28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762871.944867:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968032 is committed 00002000:00010000:3.0:1789762871.944886:0:14913:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:3.0:1789762871.944893:0:14913:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb mtime from disk: 0 -> 1789762872 00002000:00010000:3.0:1789762871.944896:0:14913:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb ctime from disk: 0 -> 1789762872 00002000:00010000:3.0:1789762871.944899:0:14913:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:3.0:1789762871.944912:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04c28 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.944922:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04c28 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.944927:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04c28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.944932:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4c600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762871.944935:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04c28 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.944942:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04c28 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.944956:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04c28 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.944961:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04c28 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762871.945021:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015065344:12345-192.168.203.48@tcp:4:dd.0 Request processed in 16507us (16631us total) trans 4294968032 rc 0/0 00000100:00100000:3.0:1789762871.945029:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 926 00000100:00100000:3.0:1789762871.945038:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 2 deadline 354401252788 time 395232413271 state 0x2 ref 0 00000100:00100000:0.0:1789762871.947029:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762871.947115:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015065472 00000100:00100000:1.0:1789762871.947123:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015065472 00000100:00100000:1.0:1789762871.947157:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 2 deadline 354401252788 time 395232413271 state 0x0 ref 1 00000100:00100000:1.0:1789762871.947168:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 2 deadline 395282413271 time 395232413271 state 0x1 ref 1 00000100:00100000:1.0:1789762871.947190:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 1 deadline 395282413271 time 395251098780 state 0x0 ref 1 00000100:00100000:1.0:1789762871.947200:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015065472:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762871.947224:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 1048576, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x803 00010000:00010000:1.0:1789762871.947229:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04c2f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.947239:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:1.0:1789762871.947243:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762871.947247:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04c2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.947255:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04c2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762871.959320:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968033 is committed 00002000:00010000:3.0:1789762871.959377:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:3.0:1789762871.959382:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:3.0:1789762871.959390:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04c2f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.959402:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04c2f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.959411:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04c2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.959419:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762871.959423:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04c2f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.959433:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04c2f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.959449:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04c2f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.959460:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04c2f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762871.959507:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015065472:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12307us (12478us total) trans 4294968033 rc 0/0 00000100:00100000:3.0:1789762871.959514:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 927 00000100:00100000:3.0:1789762871.959521:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 1 deadline 395282413271 time 395251098780 state 0x2 ref 0 00000100:00100000:0.0:1789762871.961359:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762871.961411:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015065600 00000100:00100000:1.0:1789762871.961417:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015065600 00000100:00100000:1.0:1789762871.961454:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 1 deadline 395282413271 time 395251098780 state 0x0 ref 1 00000100:00100000:1.0:1789762871.961464:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 1 deadline 395301098780 time 395251098780 state 0x1 ref 1 00000100:00100000:1.0:1789762871.961482:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395301098780 time 395265391288 state 0x0 ref 1 00000100:00100000:1.0:1789762871.961490:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015065600:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762871.961510:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 2097152, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x1007 00010000:00010000:1.0:1789762871.961516:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04c36 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.961525:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:1.0:1789762871.961528:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762871.961531:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04c36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762871.961539:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04c36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762871.973177:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968034 is committed 00002000:00010000:3.0:1789762871.973189:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:3.0:1789762871.973194:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:3.0:1789762871.973203:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04c36 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.973213:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04c36 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.973220:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04c36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.973225:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762871.973228:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04c36 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.973234:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04c36 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.973248:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04c36 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762871.973253:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04c36 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762871.973297:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015065600:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11807us (11938us total) trans 4294968034 rc 0/0 00000100:00100000:3.0:1789762871.973304:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 928 00000100:00100000:3.0:1789762871.973310:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395301098780 time 395265391288 state 0x2 ref 0 00000100:00100000:1.0:1789762871.975199:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762871.975550:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015065728 00000100:00100000:2.0:1789762871.975559:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015065728 00000100:00100000:2.0:1789762871.975609:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395301098780 time 395265391288 state 0x0 ref 1 00000100:00100000:2.0:1789762871.975625:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395315391288 time 395265391288 state 0x1 ref 1 00000100:00100000:1.0:1789762872.011579:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395315391288 time 395315477559 state 0x0 ref 1 00000100:00100000:1.0:1789762872.011600:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015065728:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762872.011646:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 3145728, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x1807 00010000:00010000:1.0:1789762872.011653:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04c3d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.011665:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:1.0:1789762872.011669:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762872.011674:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04c3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762872.011684:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04c3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762872.024035:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968035 is committed 00002000:00010000:3.0:1789762872.024053:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:3.0:1789762872.024059:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:3.0:1789762872.024070:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04c3d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.024080:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04c3d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.024100:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04c3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.024106:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.024110:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04c3d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.024121:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04c3d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.024135:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04c3d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.024140:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04c3d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.024199:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015065728:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12605us (49000us total) trans 4294968035 rc 0/0 00000100:00100000:3.0:1789762872.024207:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 929 00000100:00100000:3.0:1789762872.024217:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395315391288 time 395315477559 state 0x2 ref 0 00000100:00100000:0.0:1789762872.026141:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762872.026174:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015065856 00000100:00100000:1.0:1789762872.026182:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015065856 00000100:00100000:1.0:1789762872.026219:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395315391288 time 395315477559 state 0x0 ref 1 00000100:00100000:1.0:1789762872.026230:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395365477559 time 395315477559 state 0x1 ref 1 00000100:00100000:0.0:1789762872.061695:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395365477559 time 395365589351 state 0x0 ref 1 00000100:00100000:0.0:1789762872.061724:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015065856:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762872.061779:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 4194304, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x2007 00010000:00010000:0.0:1789762872.061789:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b04c44 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.061833:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:0.0:1789762872.061838:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762872.061845:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b04c44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762872.061860:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b04c44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762872.073574:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968036 is committed 00002000:00010000:3.0:1789762872.073583:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:3.0:1789762872.073589:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:3.0:1789762872.073601:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b04c44 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.073611:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b04c44 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.073617:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b04c44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.073627:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.073630:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b04c44 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.073637:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b04c44 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.073651:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b04c44 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.073656:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c03d1e59/0x416193e012b04c44 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.073724:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015065856:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12008us (47582us total) trans 4294968036 rc 0/0 00000100:00100000:3.0:1789762872.073733:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 930 00000100:00100000:3.0:1789762872.073742:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395365477559 time 395365589351 state 0x2 ref 0 00000100:00100000:1.0:1789762872.075716:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762872.075751:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015065984 00000100:00100000:3.0:1789762872.075758:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015065984 00000100:00100000:3.0:1789762872.075795:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395365477559 time 395365589351 state 0x0 ref 1 00000100:00100000:3.0:1789762872.075805:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395415589351 time 395365589351 state 0x1 ref 1 00000100:00100000:3.0:1789762872.111762:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395415589351 time 395415658184 state 0x0 ref 1 00000100:00100000:3.0:1789762872.111787:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015065984:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.111849:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 5242880, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x2807 00010000:00010000:3.0:1789762872.111857:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c4b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.111870:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.111874:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.111879:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.111890:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762872.124455:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968037 is committed 00002000:00010000:3.0:1789762872.124482:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:3.0:1789762872.124487:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:3.0:1789762872.124499:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c4b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.124510:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c4b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.124517:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.124522:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.124526:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c4b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.124533:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c4b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.124548:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c4b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.124553:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c4b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.124617:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015065984:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12838us (48899us total) trans 4294968037 rc 0/0 00000100:00100000:3.0:1789762872.124625:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 931 00000100:00100000:3.0:1789762872.124635:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395415589351 time 395415658184 state 0x2 ref 0 00000100:00100000:1.0:1789762872.126544:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762872.126585:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015066112 00000100:00100000:2.0:1789762872.126596:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015066112 00000100:00100000:2.0:1789762872.126642:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395415589351 time 395415658184 state 0x0 ref 1 00000100:00100000:2.0:1789762872.126653:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395465658184 time 395415658184 state 0x1 ref 1 00000100:00100000:3.0:1789762872.161877:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395465658184 time 395465775668 state 0x0 ref 1 00000100:00100000:3.0:1789762872.161899:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015066112:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.161941:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 6291456, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x3007 00010000:00010000:3.0:1789762872.161948:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c52 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.161961:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.161965:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.161970:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.161980:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762872.173795:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968038 is committed 00002000:00010000:3.0:1789762872.173858:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:3.0:1789762872.173863:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:3.0:1789762872.173873:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c52 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.173882:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c52 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.173889:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.173894:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.173898:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c52 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.173905:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c52 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.173918:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c52 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.173922:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c52 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.173980:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015066112:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12087us (47436us total) trans 4294968038 rc 0/0 00000100:00100000:3.0:1789762872.173988:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 932 00000100:00100000:3.0:1789762872.173997:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395465658184 time 395465775668 state 0x2 ref 0 00000100:00100000:0.0:1789762872.176153:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762872.176199:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015066240 00000100:00100000:1.0:1789762872.176207:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015066240 00000100:00100000:1.0:1789762872.176245:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395465658184 time 395465775668 state 0x0 ref 1 00000100:00100000:1.0:1789762872.176258:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395515775668 time 395465775668 state 0x1 ref 1 00000100:00100000:3.0:1789762872.211950:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395515775668 time 395515850479 state 0x0 ref 1 00000100:00100000:3.0:1789762872.211967:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015066240:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.212005:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 7340032, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x3807 00010000:00010000:3.0:1789762872.212011:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c59 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.212031:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.212035:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.212040:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.212050:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762872.225337:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968039 is committed 00002000:00010000:3.0:1789762872.225400:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:3.0:1789762872.225405:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:3.0:1789762872.225415:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c59 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.225432:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c59 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.225442:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.225451:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.225456:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c59 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.225466:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c59 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.225484:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c59 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.225490:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c59 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.225548:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015066240:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13586us (49395us total) trans 4294968039 rc 0/0 00000100:00100000:3.0:1789762872.225557:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 933 00000100:00100000:3.0:1789762872.225565:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395515775668 time 395515850479 state 0x2 ref 0 00000100:00100000:1.0:1789762872.227638:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762872.227669:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015066368 00000100:00100000:3.0:1789762872.227677:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015066368 00000100:00100000:3.0:1789762872.227710:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395515775668 time 395515850479 state 0x0 ref 1 00000100:00100000:3.0:1789762872.227720:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395565850479 time 395515850479 state 0x1 ref 1 00010000:00010000:0.0:1789762872.247160:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762872.261998:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395565850479 time 395565899231 state 0x0 ref 1 00000100:00100000:3.0:1789762872.262017:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015066368:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.262057:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 8388608, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x4007 00010000:00010000:3.0:1789762872.262064:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c60 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.262076:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.262079:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.262084:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.262116:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762872.274519:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968040 is committed 00002000:00010000:3.0:1789762872.274585:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:3.0:1789762872.274589:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:3.0:1789762872.274599:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c60 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.274608:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c60 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.274615:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.274621:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.274624:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c60 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.274630:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c60 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.274643:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c60 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.274648:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c60 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.274708:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015066368:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12696us (47068us total) trans 4294968040 rc 0/0 00000100:00100000:3.0:1789762872.274717:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 934 00000100:00100000:3.0:1789762872.274726:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395565850479 time 395565899231 state 0x2 ref 0 00000100:00100000:1.0:1789762872.276999:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762872.277039:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015066496 00000100:00100000:2.0:1789762872.277047:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015066496 00000100:00100000:2.0:1789762872.277110:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395565850479 time 395565899231 state 0x0 ref 1 00000100:00100000:2.0:1789762872.277122:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395615899231 time 395565899231 state 0x1 ref 1 00000100:00100000:3.0:1789762872.312104:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395615899231 time 395615986136 state 0x0 ref 1 00000100:00100000:3.0:1789762872.312132:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015066496:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.312188:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 9437184, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x4807 00010000:00010000:3.0:1789762872.312204:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c67 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.312220:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.312226:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.312233:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.312250:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762872.324622:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968041 is committed 00002000:00010000:3.0:1789762872.324630:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:3.0:1789762872.324636:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 18439 -> 20487 00010000:00010000:3.0:1789762872.324646:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c67 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.324656:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c67 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.324663:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.324669:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.324672:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c67 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.324683:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c67 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.324697:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c67 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.324702:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c67 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.324760:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015066496:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12638us (47762us total) trans 4294968041 rc 0/0 00000100:00100000:3.0:1789762872.324769:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 935 00000100:00100000:3.0:1789762872.324778:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395615899231 time 395615986136 state 0x2 ref 0 00000100:00100000:0.0:1789762872.326996:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762872.327041:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015066624 00000100:00100000:1.0:1789762872.327050:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015066624 00000100:00100000:1.0:1789762872.327103:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395615899231 time 395615986136 state 0x0 ref 1 00000100:00100000:1.0:1789762872.327116:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395665986136 time 395615986136 state 0x1 ref 1 00000100:00100000:3.0:1789762872.362157:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395665986136 time 395666053373 state 0x0 ref 1 00000100:00100000:3.0:1789762872.362183:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015066624:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.362225:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 10485760, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x5007 00010000:00010000:3.0:1789762872.362231:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c6e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.362243:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.362247:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.362251:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.362262:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762872.376666:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968042 is committed 00002000:00010000:3.0:1789762872.376732:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:3.0:1789762872.376736:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:3.0:1789762872.376746:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c6e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.376755:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c6e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.376762:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.376766:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.376770:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c6e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.376777:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c6e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.376790:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c6e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.376794:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c6e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.376867:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015066624:12345-192.168.203.48@tcp:4:dd.0 Request processed in 14691us (49870us total) trans 4294968042 rc 0/0 00000100:00100000:3.0:1789762872.376875:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 936 00000100:00100000:3.0:1789762872.376884:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395665986136 time 395666053373 state 0x2 ref 0 00000100:00100000:1.0:1789762872.379027:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762872.379061:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015066752 00000100:00100000:3.0:1789762872.379070:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015066752 00000100:00100000:3.0:1789762872.379121:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395665986136 time 395666053373 state 0x0 ref 1 00000100:00100000:3.0:1789762872.379132:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395716053373 time 395666053373 state 0x1 ref 1 00000100:00100000:3.0:1789762872.412196:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395716053373 time 395716093494 state 0x0 ref 1 00000100:00100000:3.0:1789762872.412217:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015066752:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.412263:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 11534336, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x5807 00010000:00010000:3.0:1789762872.412271:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c75 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.412284:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.412288:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.412292:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.412304:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762872.423841:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968043 is committed 00002000:00010000:3.0:1789762872.423866:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 11534336 -> 12582912 00002000:00010000:3.0:1789762872.423872:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 22535 -> 24583 00010000:00010000:3.0:1789762872.423885:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c75 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.423899:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c75 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.423909:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.423917:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.423923:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c75 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.423934:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c75 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.423950:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c75 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.423956:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c75 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.424029:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015066752:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11815us (45000us total) trans 4294968043 rc 0/0 00000100:00100000:3.0:1789762872.424039:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 937 00000100:00100000:3.0:1789762872.424049:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395716053373 time 395716093494 state 0x2 ref 0 00000100:00100000:1.0:1789762872.426270:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762872.426349:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015066880 00000100:00100000:2.0:1789762872.426358:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015066880 00000100:00100000:2.0:1789762872.426397:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395716053373 time 395716093494 state 0x0 ref 1 00000100:00100000:2.0:1789762872.426408:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395766093494 time 395716093494 state 0x1 ref 1 00000001:00080000:0.0:1789762872.447275:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967566 is committed 00000001:00080000:0.0:1789762872.447289:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967567 is committed 00000100:00100000:3.0:1789762872.462274:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395766093494 time 395766171648 state 0x0 ref 1 00000100:00100000:3.0:1789762872.462296:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015066880:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.462344:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 12582912, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x6007 00010000:00010000:3.0:1789762872.462351:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c7c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.462365:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.462369:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.462375:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.462391:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762872.473871:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968044 is committed 00002000:00010000:3.0:1789762872.473888:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 12582912 -> 13631488 00002000:00010000:3.0:1789762872.473894:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:3.0:1789762872.473905:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c7c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.473914:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c7c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.473926:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.473931:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.473934:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c7c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.473941:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c7c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.473953:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c7c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.473958:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c7c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.474014:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015066880:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11724us (47744us total) trans 4294968044 rc 0/0 00000100:00100000:3.0:1789762872.474021:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 938 00000100:00100000:3.0:1789762872.474030:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395766093494 time 395766171648 state 0x2 ref 0 00000100:00100000:1.0:1789762872.475880:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762872.475913:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015067008 00000100:00100000:3.0:1789762872.475920:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015067008 00000100:00100000:3.0:1789762872.475954:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395766093494 time 395766171648 state 0x0 ref 1 00000100:00100000:3.0:1789762872.475967:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395816171648 time 395766171648 state 0x1 ref 1 00000100:00100000:3.0:1789762872.512346:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395816171648 time 395816238426 state 0x0 ref 1 00000100:00100000:3.0:1789762872.512375:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015067008:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.512428:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 13631488, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x6807 00010000:00010000:3.0:1789762872.512437:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c83 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.512453:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.512458:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.512464:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.512479:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762872.525068:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968045 is committed 00002000:00010000:3.0:1789762872.525131:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:3.0:1789762872.525137:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:3.0:1789762872.525149:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c83 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.525160:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c83 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.525176:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.525184:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.525189:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c83 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.525200:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c83 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.525217:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c83 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.525226:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c83 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.525298:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015067008:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12930us (49414us total) trans 4294968045 rc 0/0 00000100:00100000:3.0:1789762872.525309:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 939 00000100:00100000:3.0:1789762872.525321:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395816171648 time 395816238426 state 0x2 ref 0 00000100:00100000:0.0:1789762872.527414:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762872.527454:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015067136 00000100:00100000:3.0:1789762872.527463:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015067136 00000100:00100000:3.0:1789762872.527503:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395816171648 time 395816238426 state 0x0 ref 1 00000100:00100000:3.0:1789762872.527521:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395866238426 time 395816238426 state 0x1 ref 1 00000100:00100000:3.0:1789762872.562392:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395866238426 time 395866294163 state 0x0 ref 1 00000100:00100000:3.0:1789762872.562409:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015067136:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.562446:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 14680064, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x7007 00010000:00010000:3.0:1789762872.562452:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c8a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.562463:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.562466:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.562470:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.562480:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762872.574915:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968046 is committed 00002000:00010000:3.0:1789762872.574915:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:3.0:1789762872.574920:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:3.0:1789762872.574931:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c8a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.574940:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c8a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.574952:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.574957:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.574961:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c8a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.574967:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c8a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.574979:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c8a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.574984:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c8a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.575032:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015067136:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12627us (47618us total) trans 4294968046 rc 0/0 00000100:00100000:3.0:1789762872.575040:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 940 00000100:00100000:3.0:1789762872.575048:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395866238426 time 395866294163 state 0x2 ref 0 00000100:00100000:1.0:1789762872.577136:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762872.577191:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015067264 00000100:00100000:2.0:1789762872.577199:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015067264 00000100:00100000:2.0:1789762872.577238:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395866238426 time 395866294163 state 0x0 ref 1 00000100:00100000:2.0:1789762872.577253:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395916294163 time 395866294163 state 0x1 ref 1 00000100:00100000:3.0:1789762872.612511:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395916294163 time 395916411334 state 0x0 ref 1 00000100:00100000:3.0:1789762872.612529:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015067264:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.612568:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 15728640, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x7807 00010000:00010000:3.0:1789762872.612574:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c91 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.612585:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.612589:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.612594:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.612604:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762872.624607:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968047 is committed 00002000:00010000:3.0:1789762872.624666:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:3.0:1789762872.624671:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:3.0:1789762872.624681:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c91 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.624691:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c91 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.624697:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.624702:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.624705:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c91 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.624717:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c91 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.624730:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c91 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.624735:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c91 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.624795:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015067264:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12269us (47657us total) trans 4294968047 rc 0/0 00000100:00100000:3.0:1789762872.624802:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 941 00000100:00100000:3.0:1789762872.624811:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395916294163 time 395916411334 state 0x2 ref 0 00000100:00100000:1.0:1789762872.626927:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762872.626962:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015067392 00000100:00100000:3.0:1789762872.626969:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015067392 00000100:00100000:3.0:1789762872.627005:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395916294163 time 395916411334 state 0x0 ref 1 00000100:00100000:3.0:1789762872.627016:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395966411334 time 395916411334 state 0x1 ref 1 00000100:00100000:3.0:1789762872.662569:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395966411334 time 395966469588 state 0x0 ref 1 00000100:00100000:3.0:1789762872.662588:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015067392:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.662627:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 16777216, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x8007 00010000:00010000:3.0:1789762872.662634:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c98 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.662645:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=395s v=(0 0) 00010000:00010000:3.0:1789762872.662649:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.662653:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.662663:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762872.678118:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968048 is committed 00002000:00010000:3.0:1789762872.678120:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 16777216 -> 17825792 00002000:00010000:3.0:1789762872.678125:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:3.0:1789762872.678136:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c98 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.678145:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c98 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.678151:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.678156:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.678160:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c98 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.678167:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c98 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.678180:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c98 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.678186:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c98 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.678239:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015067392:12345-192.168.203.48@tcp:4:dd.0 Request processed in 15655us (51311us total) trans 4294968048 rc 0/0 00000100:00100000:3.0:1789762872.678246:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 942 00000100:00100000:3.0:1789762872.678254:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395966411334 time 395966469588 state 0x2 ref 0 00000100:00100000:0.0:1789762872.680328:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762872.680360:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015067520 00000100:00100000:3.0:1789762872.680368:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015067520 00000100:00100000:3.0:1789762872.680402:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 395966411334 time 395966469588 state 0x0 ref 1 00000100:00100000:3.0:1789762872.680412:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396016469588 time 395966469588 state 0x1 ref 1 00000100:00100000:3.0:1789762872.712618:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396016469588 time 396016515917 state 0x0 ref 1 00000100:00100000:3.0:1789762872.712640:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015067520:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.712686:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 17825792, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x8807 00010000:00010000:3.0:1789762872.712693:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c9f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.712709:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762872.712713:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.712717:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.712729:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762872.725828:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968049 is committed 00002000:00010000:3.0:1789762872.725835:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:3.0:1789762872.725841:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:3.0:1789762872.725852:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c9f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.725862:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c9f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.725870:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.725874:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.725878:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c9f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.725885:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c9f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.725904:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c9f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.725909:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04c9f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.725971:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015067520:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13337us (45641us total) trans 4294968049 rc 0/0 00000100:00100000:3.0:1789762872.725979:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 943 00000100:00100000:3.0:1789762872.725988:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396016469588 time 396016515917 state 0x2 ref 0 00000100:00100000:0.0:1789762872.728241:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762872.728294:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015067648 00000100:00100000:2.0:1789762872.728303:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015067648 00000100:00100000:2.0:1789762872.728347:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396016469588 time 396016515917 state 0x0 ref 1 00000100:00100000:2.0:1789762872.728375:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396066515917 time 396016515917 state 0x1 ref 1 00000100:00100000:3.0:1789762872.762686:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396066515917 time 396066584698 state 0x0 ref 1 00000100:00100000:3.0:1789762872.762705:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015067648:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.762749:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 18874368, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x9007 00010000:00010000:3.0:1789762872.762756:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ca6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.762768:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762872.762772:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.762776:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ca6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.762787:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ca6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762872.774958:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968050 is committed 00002000:00010000:3.0:1789762872.774968:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:3.0:1789762872.774974:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:3.0:1789762872.774985:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ca6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.774994:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ca6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.775000:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ca6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.775006:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.775009:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ca6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.775016:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ca6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.775029:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ca6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.775034:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ca6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.775121:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015067648:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12420us (46878us total) trans 4294968050 rc 0/0 00000100:00100000:3.0:1789762872.775130:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 944 00000100:00100000:3.0:1789762872.775138:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396066515917 time 396066584698 state 0x2 ref 0 00000100:00100000:0.0:1789762872.777248:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762872.777284:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015067776 00000100:00100000:3.0:1789762872.777292:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015067776 00000100:00100000:3.0:1789762872.777328:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396066515917 time 396066584698 state 0x0 ref 1 00000100:00100000:3.0:1789762872.777338:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396116584698 time 396066584698 state 0x1 ref 1 00000100:00100000:3.0:1789762872.812756:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396116584698 time 396116649059 state 0x0 ref 1 00000100:00100000:3.0:1789762872.812784:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015067776:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.812850:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 19922944, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0x9807 00010000:00010000:3.0:1789762872.812861:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cad lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.812876:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762872.812882:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.812888:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.812903:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762872.826027:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968051 is committed 00002000:00010000:3.0:1789762872.826051:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 19922944 -> 20971520 00002000:00010000:3.0:1789762872.826058:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:3.0:1789762872.826068:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cad lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.826080:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.826110:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.826119:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.826124:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.826135:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.826152:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.826162:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.826229:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015067776:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13452us (48980us total) trans 4294968051 rc 0/0 00000100:00100000:3.0:1789762872.826240:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 945 00000100:00100000:3.0:1789762872.826251:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396116584698 time 396116649059 state 0x2 ref 0 00000100:00100000:0.0:1789762872.828304:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762872.828333:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015067904 00000100:00100000:3.0:1789762872.828341:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015067904 00000100:00100000:3.0:1789762872.828381:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396116584698 time 396116649059 state 0x0 ref 1 00000100:00100000:3.0:1789762872.828396:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396166649059 time 396116649059 state 0x1 ref 1 00000100:00100000:3.0:1789762872.862806:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396166649059 time 396166706535 state 0x0 ref 1 00000100:00100000:3.0:1789762872.862839:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015067904:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.862887:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 20971520, mtime: 0x6aad9d38, atime: 0x0, ctime: 0x6aad9d38, blocks: 0xa007 00010000:00010000:3.0:1789762872.862895:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.862908:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762872.862913:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.862920:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.862942:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762872.875615:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968052 is committed 00002000:00010000:3.0:1789762872.875678:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:3.0:1789762872.875683:0:12582:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb mtime from disk: 1789762872 -> 1789762873 00002000:00010000:3.0:1789762872.875685:0:12582:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb ctime from disk: 1789762872 -> 1789762873 00002000:00010000:3.0:1789762872.875687:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:3.0:1789762872.875697:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.875705:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.875712:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.875717:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.875720:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.875727:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.875749:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.875753:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.875806:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015067904:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12973us (47501us total) trans 4294968052 rc 0/0 00000100:00100000:3.0:1789762872.875827:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 946 00000100:00100000:3.0:1789762872.875837:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396166649059 time 396166706535 state 0x2 ref 0 00000100:00100000:1.0:1789762872.878487:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762872.878546:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015068032 00000100:00100000:2.0:1789762872.878555:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015068032 00000100:00100000:2.0:1789762872.878594:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396166649059 time 396166706535 state 0x0 ref 1 00000100:00100000:2.0:1789762872.878609:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396216706535 time 396166706535 state 0x1 ref 1 00000100:00100000:3.0:1789762872.912891:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396216706535 time 396216789776 state 0x0 ref 1 00000100:00100000:3.0:1789762872.912910:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015068032:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.912952:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 22020096, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0xa807 00010000:00010000:3.0:1789762872.912959:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cbb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.912971:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762872.912975:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.912980:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.912991:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762872.925419:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968053 is committed 00002000:00010000:3.0:1789762872.925434:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 22020096 -> 23068672 00002000:00010000:3.0:1789762872.925440:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:3.0:1789762872.925451:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cbb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.925462:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cbb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.925469:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.925479:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.925483:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cbb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.925489:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cbb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.925502:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cbb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.925507:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cbb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.925560:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015068032:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12654us (47073us total) trans 4294968053 rc 0/0 00000100:00100000:3.0:1789762872.925569:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 947 00000100:00100000:3.0:1789762872.925577:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396216706535 time 396216789776 state 0x2 ref 0 00000100:00100000:0.0:1789762872.927716:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762872.927750:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015068160 00000100:00100000:3.0:1789762872.927758:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015068160 00000100:00100000:3.0:1789762872.927792:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396216706535 time 396216789776 state 0x0 ref 1 00000100:00100000:3.0:1789762872.927802:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396266789776 time 396216789776 state 0x1 ref 1 00000100:00100000:3.0:1789762872.962926:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396266789776 time 396266828087 state 0x0 ref 1 00000100:00100000:3.0:1789762872.962942:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015068160:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762872.962974:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 23068672, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0xb007 00010000:00010000:3.0:1789762872.962980:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.962990:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762872.962994:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762872.962998:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.963008:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762872.974242:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968054 is committed 00002000:00010000:3.0:1789762872.974265:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:3.0:1789762872.974270:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:3.0:1789762872.974280:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.974289:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.974295:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.974300:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762872.974304:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.974311:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.974322:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762872.974327:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762872.974376:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831aad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015068160:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11437us (46660us total) trans 4294968054 rc 0/0 00000100:00100000:3.0:1789762872.974385:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 948 00000100:00100000:3.0:1789762872.974392:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396266789776 time 396266828087 state 0x2 ref 0 00000100:00100000:1.0:1789762872.976784:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762872.976830:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015068288 00000100:00100000:3.0:1789762872.976838:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015068288 00000100:00100000:3.0:1789762872.976867:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396266789776 time 396266828087 state 0x0 ref 1 00000100:00100000:3.0:1789762872.976877:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396316828087 time 396266828087 state 0x1 ref 1 00000100:00100000:3.0:1789762873.012979:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396316828087 time 396316876991 state 0x0 ref 1 00000100:00100000:3.0:1789762873.013014:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015068288:12345-192.168.203.48@tcp:4:dd.0 00000100:00080000:0.0:1789762873.013051:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789762873.013067:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 24117248, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0xb807 00010000:00010000:3.0:1789762873.013076:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.013108:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.013112:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.013117:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.013128:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.025136:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968055 is committed 00002000:00010000:3.0:1789762873.025140:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 24117248 -> 25165824 00002000:00010000:3.0:1789762873.025147:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:3.0:1789762873.025160:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.025174:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.025185:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.025193:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.025199:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.025216:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.025233:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.025239:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cc9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.025298:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015068288:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12304us (48514us total) trans 4294968055 rc 0/0 00000100:00100000:3.0:1789762873.025307:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 949 00000100:00100000:3.0:1789762873.025315:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396316828087 time 396316876991 state 0x2 ref 0 00000100:00100000:0.0:1789762873.027706:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762873.027781:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015068416 00000100:00100000:2.0:1789762873.027790:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015068416 00000100:00100000:2.0:1789762873.027843:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396316828087 time 396316876991 state 0x0 ref 1 00000100:00100000:2.0:1789762873.027854:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396366876991 time 396316876991 state 0x1 ref 1 00000100:00100000:3.0:1789762873.063061:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396366876991 time 396366959918 state 0x0 ref 1 00000100:00100000:3.0:1789762873.063082:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015068416:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.063157:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 25165824, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0xc007 00010000:00010000:3.0:1789762873.063164:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.063177:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.063180:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.063185:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.063196:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762873.074197:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:3.0:1789762873.074203:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:3.0:1789762873.074212:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.074213:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968056 is committed 00010000:00010000:3.0:1789762873.074226:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.074234:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.074239:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.074242:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.074248:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.074260:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.074265:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.074320:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831aa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015068416:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11243us (46613us total) trans 4294968056 rc 0/0 00000100:00100000:3.0:1789762873.074328:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 950 00000100:00100000:3.0:1789762873.074336:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396366876991 time 396366959918 state 0x2 ref 0 00000100:00100000:1.0:1789762873.076664:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.076695:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015068544 00000100:00100000:3.0:1789762873.076704:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015068544 00000100:00100000:3.0:1789762873.076736:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396366876991 time 396366959918 state 0x0 ref 1 00000100:00100000:3.0:1789762873.076746:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396416959918 time 396366959918 state 0x1 ref 1 00000100:00100000:3.0:1789762873.113125:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396416959918 time 396417023731 state 0x0 ref 1 00000100:00100000:3.0:1789762873.113145:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015068544:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.113185:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 26214400, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0xc807 00010000:00010000:3.0:1789762873.113193:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.113205:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.113208:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.113213:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.113223:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.125485:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968057 is committed 00002000:00010000:3.0:1789762873.125490:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 26214400 -> 27262976 00002000:00010000:3.0:1789762873.125496:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:3.0:1789762873.125509:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.125521:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.125530:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.125536:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.125539:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.125546:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.125559:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.125565:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cd7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.125617:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015068544:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12478us (48952us total) trans 4294968057 rc 0/0 00000100:00100000:3.0:1789762873.125626:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 951 00000100:00100000:3.0:1789762873.125635:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396416959918 time 396417023731 state 0x2 ref 0 00000100:00100000:0.0:1789762873.128123:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.128148:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015068672 00000100:00100000:3.0:1789762873.128156:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015068672 00000100:00100000:3.0:1789762873.128196:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396416959918 time 396417023731 state 0x0 ref 1 00000100:00100000:3.0:1789762873.128208:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396467023731 time 396417023731 state 0x1 ref 1 00000100:00100000:3.0:1789762873.163163:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396467023731 time 396467066797 state 0x0 ref 1 00000100:00100000:3.0:1789762873.163179:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015068672:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.163211:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 27262976, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0xd007 00010000:00010000:3.0:1789762873.163216:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cde lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.163227:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.163231:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.163235:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.163245:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.177895:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968058 is committed 00002000:00010000:3.0:1789762873.177916:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:3.0:1789762873.177922:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:3.0:1789762873.177932:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cde lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.177941:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cde lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.177946:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.177951:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.177954:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cde lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.177961:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cde lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.177973:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cde lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.177980:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cde lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.178054:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831aa680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015068672:12345-192.168.203.48@tcp:4:dd.0 Request processed in 14877us (49931us total) trans 4294968058 rc 0/0 00000100:00100000:3.0:1789762873.178062:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 952 00000100:00100000:3.0:1789762873.178070:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396467023731 time 396467066797 state 0x2 ref 0 00000100:00100000:0.0:1789762873.180320:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762873.180377:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015068800 00000100:00100000:2.0:1789762873.180385:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015068800 00000100:00100000:2.0:1789762873.180420:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396467023731 time 396467066797 state 0x0 ref 1 00000100:00100000:2.0:1789762873.180430:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396517066797 time 396467066797 state 0x1 ref 1 00000100:00100000:3.0:1789762873.213271:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396517066797 time 396517174369 state 0x0 ref 1 00000100:00100000:3.0:1789762873.213289:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015068800:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.213326:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 28311552, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0xd807 00010000:00010000:3.0:1789762873.213334:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ce5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.213348:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.213353:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.213359:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ce5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.213371:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ce5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.224318:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968059 is committed 00002000:00010000:3.0:1789762873.224318:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:3.0:1789762873.224325:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 55303 -> 57351 00010000:00010000:3.0:1789762873.224334:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ce5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.224344:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ce5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.224350:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ce5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.224360:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.224363:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ce5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.224372:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ce5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.224388:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ce5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.224396:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04ce5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.224454:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015068800:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11168us (44133us total) trans 4294968059 rc 0/0 00000100:00100000:3.0:1789762873.224465:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 953 00000100:00100000:3.0:1789762873.224475:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396517066797 time 396517174369 state 0x2 ref 0 00000100:00100000:1.0:1789762873.226734:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.226767:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015068928 00000100:00100000:3.0:1789762873.226774:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015068928 00000100:00100000:3.0:1789762873.226808:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396517066797 time 396517174369 state 0x0 ref 1 00000100:00100000:3.0:1789762873.226835:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396567174369 time 396517174369 state 0x1 ref 1 00000100:00100000:3.0:1789762873.263323:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396567174369 time 396567226270 state 0x0 ref 1 00000100:00100000:3.0:1789762873.263339:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015068928:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.263372:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 29360128, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0xe007 00010000:00010000:3.0:1789762873.263381:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cec lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.263395:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.263399:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.263406:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.263420:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762873.271523:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789762873.274720:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968060 is committed 00002000:00010000:3.0:1789762873.274734:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:3.0:1789762873.274739:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 57351 -> 59399 00010000:00010000:3.0:1789762873.274749:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cec lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.274758:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cec lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.274764:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.274769:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.274773:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cec lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.274779:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cec lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.274791:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cec lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.274796:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cec lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.274860:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015068928:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11523us (48125us total) trans 4294968060 rc 0/0 00000100:00100000:3.0:1789762873.274867:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 954 00000100:00100000:3.0:1789762873.274874:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396567174369 time 396567226270 state 0x2 ref 0 00000100:00100000:1.0:1789762873.276898:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.276935:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015069056 00000100:00100000:3.0:1789762873.276942:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015069056 00000100:00100000:3.0:1789762873.276972:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396567174369 time 396567226270 state 0x0 ref 1 00000100:00100000:3.0:1789762873.276982:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396617226270 time 396567226270 state 0x1 ref 1 00000100:00100000:3.0:1789762873.313388:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396617226270 time 396617285316 state 0x0 ref 1 00000100:00100000:3.0:1789762873.313421:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069056:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.313463:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 30408704, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0xe807 00010000:00010000:3.0:1789762873.313472:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cf3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.313487:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.313493:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.313499:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.313511:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.325220:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968061 is committed 00002000:00010000:3.0:1789762873.325228:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 30408704 -> 31457280 00002000:00010000:3.0:1789762873.325234:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 59399 -> 61447 00010000:00010000:3.0:1789762873.325244:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cf3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.325254:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cf3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.325261:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.325266:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.325269:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cf3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.325280:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cf3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.325293:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cf3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.325298:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cf3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.325357:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069056:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11951us (48460us total) trans 4294968061 rc 0/0 00000100:00100000:3.0:1789762873.325364:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 955 00000100:00100000:3.0:1789762873.325373:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396617226270 time 396617285316 state 0x2 ref 0 00000100:00100000:1.0:1789762873.327539:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762873.327615:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015069184 00000100:00100000:2.0:1789762873.327624:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015069184 00000100:00100000:2.0:1789762873.327661:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396617226270 time 396617285316 state 0x0 ref 1 00000100:00100000:2.0:1789762873.327672:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396667285316 time 396617285316 state 0x1 ref 1 00000100:00100000:3.0:1789762873.363487:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396667285316 time 396667389494 state 0x0 ref 1 00000100:00100000:3.0:1789762873.363503:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015069184:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.363537:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 31457280, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0xf007 00010000:00010000:3.0:1789762873.363544:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cfa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.363555:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.363559:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.363563:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.363574:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.375013:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968062 is committed 00002000:00010000:3.0:1789762873.375022:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:3.0:1789762873.375027:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 61447 -> 63495 00010000:00010000:3.0:1789762873.375037:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cfa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.375046:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cfa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.375053:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.375058:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.375061:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cfa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.375067:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cfa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.375078:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cfa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.375083:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04cfa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.375165:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015069184:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11665us (47636us total) trans 4294968062 rc 0/0 00000100:00100000:3.0:1789762873.375173:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 956 00000100:00100000:3.0:1789762873.375181:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396667285316 time 396667389494 state 0x2 ref 0 00000100:00100000:1.0:1789762873.377502:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.377537:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015069312 00000100:00100000:3.0:1789762873.377545:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015069312 00000100:00100000:3.0:1789762873.377577:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396667285316 time 396667389494 state 0x0 ref 1 00000100:00100000:3.0:1789762873.377588:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396717389494 time 396667389494 state 0x1 ref 1 00000100:00100000:3.0:1789762873.413532:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396717389494 time 396717434728 state 0x0 ref 1 00000100:00100000:3.0:1789762873.413547:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015069312:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.413579:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 32505856, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0xf807 00010000:00010000:3.0:1789762873.413585:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d01 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.413597:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.413600:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.413605:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.413615:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.425319:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968063 is committed 00002000:00010000:3.0:1789762873.425325:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:3.0:1789762873.425330:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 63495 -> 65543 00010000:00010000:3.0:1789762873.425339:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d01 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.425349:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d01 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.425355:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.425360:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.425363:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d01 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.425370:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d01 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.425381:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d01 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.425386:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d01 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.425438:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015069312:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11891us (47934us total) trans 4294968063 rc 0/0 00000100:00100000:3.0:1789762873.425451:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 957 00000100:00100000:3.0:1789762873.425463:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396717389494 time 396717434728 state 0x2 ref 0 00000100:00100000:0.0:1789762873.427602:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.427635:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015069440 00000100:00100000:3.0:1789762873.427641:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015069440 00000100:00100000:3.0:1789762873.427669:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396717389494 time 396717434728 state 0x0 ref 1 00000100:00100000:3.0:1789762873.427680:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396767434728 time 396717434728 state 0x1 ref 1 00000100:00100000:3.0:1789762873.463580:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396767434728 time 396767476344 state 0x0 ref 1 00000100:00100000:3.0:1789762873.463602:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069440:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.463646:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 33554432, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0x10007 00010000:00010000:3.0:1789762873.463654:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d08 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.463667:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.463671:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.463675:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.463685:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.475049:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968064 is committed 00002000:00010000:3.0:1789762873.475049:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:3.0:1789762873.475056:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 65543 -> 67591 00010000:00010000:3.0:1789762873.475067:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d08 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.475076:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d08 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.475083:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.475104:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.475108:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d08 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.475115:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d08 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.475128:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d08 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.475133:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d08 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.475188:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069440:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11592us (47586us total) trans 4294968064 rc 0/0 00000100:00100000:3.0:1789762873.475195:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 958 00000100:00100000:3.0:1789762873.475203:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396767434728 time 396767476344 state 0x2 ref 0 00000100:00100000:0.0:1789762873.477223:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762873.477266:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015069568 00000100:00100000:2.0:1789762873.477275:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015069568 00000100:00100000:2.0:1789762873.477309:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396767434728 time 396767476344 state 0x0 ref 1 00000100:00100000:2.0:1789762873.477321:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396817476344 time 396767476344 state 0x1 ref 1 00000100:00100000:3.0:1789762873.513689:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396817476344 time 396817585523 state 0x0 ref 1 00000100:00100000:3.0:1789762873.513714:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069568:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.513763:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 34603008, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0x10807 00010000:00010000:3.0:1789762873.513772:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d0f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.513789:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.513794:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.513800:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.513829:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762873.525129:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:3.0:1789762873.525135:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 67591 -> 69639 00010000:00010000:3.0:1789762873.525145:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d0f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.525151:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968065 is committed 00010000:00010000:3.0:1789762873.525155:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d0f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.525164:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.525175:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.525178:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d0f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.525184:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d0f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.525198:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d0f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.525202:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d0f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.525257:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831abb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069568:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11552us (48035us total) trans 4294968065 rc 0/0 00000100:00100000:3.0:1789762873.525264:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 959 00000100:00100000:3.0:1789762873.525274:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396817476344 time 396817585523 state 0x2 ref 0 00000100:00100000:0.0:1789762873.527258:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.527297:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015069696 00000100:00100000:3.0:1789762873.527306:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015069696 00000100:00100000:3.0:1789762873.527342:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396817476344 time 396817585523 state 0x0 ref 1 00000100:00100000:3.0:1789762873.527352:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396867585523 time 396817585523 state 0x1 ref 1 00000100:00100000:3.0:1789762873.563726:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396867585523 time 396867627665 state 0x0 ref 1 00000100:00100000:3.0:1789762873.563746:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069696:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.563785:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 35651584, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0x11007 00010000:00010000:3.0:1789762873.563792:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d16 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.563802:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.563806:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.563825:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.563835:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762873.574556:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968066 is committed 00002000:00010000:3.0:1789762873.574570:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 35651584 -> 36700160 00002000:00010000:3.0:1789762873.574576:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 69639 -> 71687 00010000:00010000:3.0:1789762873.574586:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d16 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.574596:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d16 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.574602:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.574608:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.574612:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d16 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.574618:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d16 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.574630:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d16 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.574635:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d16 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.574689:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831ab800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069696:12345-192.168.203.48@tcp:4:dd.0 Request processed in 10947us (47432us total) trans 4294968066 rc 0/0 00000100:00100000:3.0:1789762873.574697:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 960 00000100:00100000:3.0:1789762873.574706:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396867585523 time 396867627665 state 0x2 ref 0 00000100:00100000:0.0:1789762873.576747:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.576781:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015069824 00000100:00100000:3.0:1789762873.576788:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015069824 00000100:00100000:3.0:1789762873.576832:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396867585523 time 396867627665 state 0x0 ref 1 00000100:00100000:3.0:1789762873.576843:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396917627665 time 396867627665 state 0x1 ref 1 00000100:00100000:3.0:1789762873.613779:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396917627665 time 396917680403 state 0x0 ref 1 00000100:00100000:3.0:1789762873.613794:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069824:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.613840:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 36700160, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0x11807 00010000:00010000:3.0:1789762873.613846:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d1d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.613857:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.613860:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.613865:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.613874:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762873.625237:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968067 is committed 00002000:00010000:3.0:1789762873.625254:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 36700160 -> 37748736 00002000:00010000:3.0:1789762873.625259:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 71687 -> 73735 00010000:00010000:3.0:1789762873.625270:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d1d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.625283:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d1d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.625293:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.625301:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.625306:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d1d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.625323:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d1d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.625338:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d1d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.625344:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d1d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.625399:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069824:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11607us (48653us total) trans 4294968067 rc 0/0 00000100:00100000:3.0:1789762873.625407:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 961 00000100:00100000:3.0:1789762873.625415:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396917627665 time 396917680403 state 0x2 ref 0 00000100:00100000:0.0:1789762873.627622:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762873.627696:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015069952 00000100:00100000:2.0:1789762873.627703:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015069952 00000100:00100000:2.0:1789762873.627738:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396917627665 time 396917680403 state 0x0 ref 1 00000100:00100000:2.0:1789762873.627749:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396967680403 time 396917680403 state 0x1 ref 1 00000100:00100000:3.0:1789762873.663880:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396967680403 time 396967780404 state 0x0 ref 1 00000100:00100000:3.0:1789762873.663900:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069952:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.663940:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 37748736, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0x12007 00010000:00010000:3.0:1789762873.663947:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d24 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.663960:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=396s v=(0 0) 00010000:00010000:3.0:1789762873.663964:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.663968:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.663980:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762873.675434:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968068 is committed 00002000:00010000:3.0:1789762873.675455:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:3.0:1789762873.675463:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 73735 -> 75783 00010000:00010000:3.0:1789762873.675475:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d24 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.675487:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d24 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.675493:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.675499:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.675502:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d24 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.675509:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d24 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.675522:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d24 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.675527:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d24 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.675584:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015069952:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11689us (47960us total) trans 4294968068 rc 0/0 00000100:00100000:3.0:1789762873.675592:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 962 00000100:00100000:3.0:1789762873.675601:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396967680403 time 396967780404 state 0x2 ref 0 00000100:00100000:1.0:1789762873.677806:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.677853:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015070080 00000100:00100000:3.0:1789762873.677862:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015070080 00000100:00100000:3.0:1789762873.677897:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 396967680403 time 396967780404 state 0x0 ref 1 00000100:00100000:3.0:1789762873.677908:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397017780404 time 396967780404 state 0x1 ref 1 00000100:00100000:3.0:1789762873.713937:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397017780404 time 397017836584 state 0x0 ref 1 00000100:00100000:3.0:1789762873.713957:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015070080:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.713998:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 38797312, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0x12807 00010000:00010000:3.0:1789762873.714005:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d2b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.714018:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:3.0:1789762873.714021:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.714025:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.714037:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.725476:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968069 is committed 00002000:00010000:3.0:1789762873.725498:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:3.0:1789762873.725504:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 75783 -> 77831 00010000:00010000:3.0:1789762873.725515:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d2b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.725524:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d2b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.725530:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.725535:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.725539:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d2b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.725546:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d2b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.725558:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d2b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.725564:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d2b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.725624:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015070080:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11672us (47816us total) trans 4294968069 rc 0/0 00000100:00100000:3.0:1789762873.725632:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 963 00000100:00100000:3.0:1789762873.725642:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397017780404 time 397017836584 state 0x2 ref 0 00000100:00100000:0.0:1789762873.727870:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.727906:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015070208 00000100:00100000:3.0:1789762873.727914:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015070208 00000100:00100000:3.0:1789762873.727950:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397017780404 time 397017836584 state 0x0 ref 1 00000100:00100000:3.0:1789762873.727961:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397067836584 time 397017836584 state 0x1 ref 1 00000100:00100000:3.0:1789762873.763984:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397067836584 time 397067887260 state 0x0 ref 1 00000100:00100000:3.0:1789762873.763999:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015070208:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.764033:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 39845888, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0x13007 00010000:00010000:3.0:1789762873.764040:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d32 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.764051:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:3.0:1789762873.764055:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.764059:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.764069:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.774427:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968070 is committed 00002000:00010000:3.0:1789762873.774445:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 39845888 -> 40894464 00002000:00010000:3.0:1789762873.774452:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 77831 -> 79879 00010000:00010000:3.0:1789762873.774464:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d32 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.774478:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d32 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.774486:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.774491:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.774494:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d32 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.774501:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d32 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.774513:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d32 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.774518:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d32 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.774572:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831ab480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015070208:12345-192.168.203.48@tcp:4:dd.0 Request processed in 10575us (46701us total) trans 4294968070 rc 0/0 00000100:00100000:3.0:1789762873.774579:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 964 00000100:00100000:3.0:1789762873.774588:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397067836584 time 397067887260 state 0x2 ref 0 00000100:00100000:0.0:1789762873.776663:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762873.776706:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015070336 00000100:00100000:2.0:1789762873.776713:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015070336 00000100:00100000:2.0:1789762873.776745:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397067836584 time 397067887260 state 0x0 ref 1 00000100:00100000:2.0:1789762873.776755:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397117887260 time 397067887260 state 0x1 ref 1 00000100:00100000:3.0:1789762873.814072:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397117887260 time 397117969071 state 0x0 ref 1 00000100:00100000:3.0:1789762873.814111:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015070336:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.814158:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 40894464, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0x13807 00010000:00010000:3.0:1789762873.814165:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d39 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.814179:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:3.0:1789762873.814183:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.814187:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.814208:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.825580:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968071 is committed 00002000:00010000:3.0:1789762873.825595:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:3.0:1789762873.825600:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 79879 -> 81927 00010000:00010000:3.0:1789762873.825611:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d39 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.825620:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d39 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.825627:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.825637:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.825641:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d39 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.825647:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d39 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.825661:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d39 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.825665:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d39 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.825721:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015070336:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11633us (49058us total) trans 4294968071 rc 0/0 00000100:00100000:3.0:1789762873.825728:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 965 00000100:00100000:3.0:1789762873.825737:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397117887260 time 397117969071 state 0x2 ref 0 00000100:00100000:0.0:1789762873.828008:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.828042:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015070464 00000100:00100000:3.0:1789762873.828050:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015070464 00000100:00100000:3.0:1789762873.828100:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397117887260 time 397117969071 state 0x0 ref 1 00000100:00100000:3.0:1789762873.828112:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397167969071 time 397117969071 state 0x1 ref 1 00000100:00100000:3.0:1789762873.864123:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397167969071 time 397168022335 state 0x0 ref 1 00000100:00100000:3.0:1789762873.864142:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015070464:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.864183:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 41943040, mtime: 0x6aad9d39, atime: 0x0, ctime: 0x6aad9d39, blocks: 0x14007 00010000:00010000:3.0:1789762873.864190:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.864210:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:3.0:1789762873.864214:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.864219:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.864229:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.876045:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968072 is committed 00002000:00010000:3.0:1789762873.876122:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 41943040 -> 42991616 00002000:00010000:3.0:1789762873.876128:0:14912:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb mtime from disk: 1789762873 -> 1789762874 00002000:00010000:3.0:1789762873.876132:0:14912:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb ctime from disk: 1789762873 -> 1789762874 00002000:00010000:3.0:1789762873.876135:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 81927 -> 83975 00010000:00010000:3.0:1789762873.876147:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.876159:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.876169:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.876179:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.876183:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.876189:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d40 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.876202:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d40 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.876208:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.876268:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015070464:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12130us (48258us total) trans 4294968072 rc 0/0 00000100:00100000:3.0:1789762873.876277:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 966 00000100:00100000:3.0:1789762873.876286:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397167969071 time 397168022335 state 0x2 ref 0 00000100:00100000:0.0:1789762873.878626:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.878663:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015070592 00000100:00100000:3.0:1789762873.878672:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015070592 00000100:00100000:3.0:1789762873.878712:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397167969071 time 397168022335 state 0x0 ref 1 00000100:00100000:3.0:1789762873.878725:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397218022335 time 397168022335 state 0x1 ref 1 00000100:00100000:3.0:1789762873.914167:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397218022335 time 397218065898 state 0x0 ref 1 00000100:00100000:3.0:1789762873.914187:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015070592:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.914229:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 42991616, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x14807 00010000:00010000:3.0:1789762873.914236:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d47 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.914247:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:3.0:1789762873.914251:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.914256:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.914266:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762873.926059:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968073 is committed 00002000:00010000:3.0:1789762873.926071:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 42991616 -> 44040192 00002000:00010000:3.0:1789762873.926077:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 83975 -> 86023 00010000:00010000:3.0:1789762873.926103:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d47 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.926113:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d47 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.926120:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.926125:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.926129:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d47 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.926136:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d47 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.926149:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d47 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.926154:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d47 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.926211:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831a9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015070592:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12029us (47584us total) trans 4294968073 rc 0/0 00000100:00100000:3.0:1789762873.926220:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 967 00000100:00100000:3.0:1789762873.926230:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397218022335 time 397218065898 state 0x2 ref 0 00000100:00100000:0.0:1789762873.928429:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762873.928485:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015070720 00000100:00100000:2.0:1789762873.928496:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015070720 00000100:00100000:2.0:1789762873.928535:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397218022335 time 397218065898 state 0x0 ref 1 00000100:00100000:2.0:1789762873.928555:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397268065898 time 397218065898 state 0x1 ref 1 00000100:00100000:3.0:1789762873.964247:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397268065898 time 397268145212 state 0x0 ref 1 00000100:00100000:3.0:1789762873.964269:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015070720:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762873.964319:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 44040192, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x15007 00010000:00010000:3.0:1789762873.964328:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d4e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.964344:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:3.0:1789762873.964348:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762873.964353:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.964364:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762873.975618:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968074 is committed 00002000:00010000:3.0:1789762873.975636:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 44040192 -> 45088768 00002000:00010000:3.0:1789762873.975641:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 86023 -> 88071 00010000:00010000:3.0:1789762873.975653:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d4e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.975663:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d4e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.975669:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.975674:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762873.975677:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d4e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.975684:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d4e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.975699:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d4e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762873.975709:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d4e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762873.975769:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831aaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015070720:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11506us (47339us total) trans 4294968074 rc 0/0 00000100:00100000:3.0:1789762873.975777:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 968 00000100:00100000:3.0:1789762873.975787:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397268065898 time 397268145212 state 0x2 ref 0 00000100:00100000:1.0:1789762873.978503:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762873.978537:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015070848 00000100:00100000:3.0:1789762873.978546:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015070848 00000100:00100000:3.0:1789762873.978584:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397268065898 time 397268145212 state 0x0 ref 1 00000100:00100000:3.0:1789762873.978594:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397318145212 time 397268145212 state 0x1 ref 1 00000100:00100000:3.0:1789762874.014288:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397318145212 time 397318190075 state 0x0 ref 1 00000100:00100000:3.0:1789762874.014305:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015070848:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762874.014343:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 45088768, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x15807 00010000:00010000:3.0:1789762874.014349:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d55 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.014361:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:3.0:1789762874.014365:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762874.014370:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.014380:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762874.026697:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968075 is committed 00002000:00010000:3.0:1789762874.026705:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 45088768 -> 46137344 00002000:00010000:3.0:1789762874.026711:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 88071 -> 90121 00010000:00010000:3.0:1789762874.026724:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d55 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.026735:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d55 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.026741:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.026746:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762874.026750:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d55 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.026757:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d55 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.026772:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d55 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.026781:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d55 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762874.026874:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015070848:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12568us (48366us total) trans 4294968075 rc 0/0 00000100:00100000:3.0:1789762874.026883:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 969 00000100:00100000:3.0:1789762874.026893:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397318145212 time 397318190075 state 0x2 ref 0 00000100:00100000:0.0:1789762874.029660:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.029694:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015070976 00000100:00100000:3.0:1789762874.029703:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015070976 00000100:00100000:3.0:1789762874.029741:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397318145212 time 397318190075 state 0x0 ref 1 00000100:00100000:3.0:1789762874.029752:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397368190075 time 397318190075 state 0x1 ref 1 00000100:00100000:3.0:1789762874.064347:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397368190075 time 397368246692 state 0x0 ref 1 00000100:00100000:3.0:1789762874.064367:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d831ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015070976:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762874.064408:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 46137344, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x16009 00010000:00010000:3.0:1789762874.064415:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d5c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.064428:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:3.0:1789762874.064431:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762874.064438:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.064452:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762874.075979:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968076 is committed 00002000:00010000:2.0:1789762874.075985:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 46137344 -> 47185920 00002000:00010000:2.0:1789762874.075991:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 90121 -> 92169 00010000:00010000:2.0:1789762874.076003:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d5c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.076013:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d5c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.076019:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.076024:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762874.076028:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d5c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.076035:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d5c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.076048:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d5c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.076054:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b04d5c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762874.076149:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d831ab100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015070976:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11786us (46488us total) trans 4294968076 rc 0/0 00000100:00100000:2.0:1789762874.076157:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 970 00000100:00100000:2.0:1789762874.076166:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397368190075 time 397368246692 state 0x2 ref 0 00000100:00100000:0.0:1789762874.078502:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762874.078539:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015071104 00000100:00100000:2.0:1789762874.078547:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015071104 00000100:00100000:2.0:1789762874.078584:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397368190075 time 397368246692 state 0x0 ref 1 00000100:00100000:2.0:1789762874.078595:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397418246692 time 397368246692 state 0x1 ref 1 00000100:00100000:3.0:1789762874.114502:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397418246692 time 397418394531 state 0x0 ref 1 00000100:00100000:3.0:1789762874.114532:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015071104:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762874.114586:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 47185920, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x16809 00010000:00010000:3.0:1789762874.114594:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d63 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.114607:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:3.0:1789762874.114610:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762874.114615:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.114625:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762874.126009:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968077 is committed 00002000:00010000:3.0:1789762874.126027:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 47185920 -> 48234496 00002000:00010000:3.0:1789762874.126047:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 92169 -> 94217 00010000:00010000:3.0:1789762874.126061:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d63 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.126077:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d63 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.126083:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.126106:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762874.126120:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d63 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.126127:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d63 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.126143:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d63 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.126149:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d63 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762874.126233:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015071104:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11710us (47730us total) trans 4294968077 rc 0/0 00000100:00100000:3.0:1789762874.126242:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 971 00000100:00100000:3.0:1789762874.126253:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397418246692 time 397418394531 state 0x2 ref 0 00000100:00100000:0.0:1789762874.129212:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.129255:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015071232 00000100:00100000:3.0:1789762874.129267:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015071232 00000100:00100000:3.0:1789762874.129320:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397418246692 time 397418394531 state 0x0 ref 1 00000100:00100000:3.0:1789762874.129334:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397468394531 time 397418394531 state 0x1 ref 1 00000100:00100000:3.0:1789762874.164556:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397468394531 time 397468456690 state 0x0 ref 1 00000100:00100000:3.0:1789762874.164574:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015071232:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762874.164615:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 48234496, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x17009 00010000:00010000:3.0:1789762874.164622:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d6a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.164634:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:3.0:1789762874.164638:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762874.164643:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.164654:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762874.176324:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968078 is committed 00002000:00010000:0.0:1789762874.176338:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 48234496 -> 49283072 00002000:00010000:0.0:1789762874.176346:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 94217 -> 96265 00010000:00010000:0.0:1789762874.176361:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d6a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.176375:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.176385:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.176403:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762874.176407:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.176414:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.176429:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.176434:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b04d6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762874.176505:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015071232:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11933us (47290us total) trans 4294968078 rc 0/0 00000100:00100000:0.0:1789762874.176513:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 972 00000100:00100000:0.0:1789762874.176523:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397468394531 time 397468456690 state 0x2 ref 0 00000100:00100000:1.0:1789762874.179043:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.179103:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015071360 00000100:00100000:3.0:1789762874.179112:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015071360 00000100:00100000:3.0:1789762874.179154:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397468394531 time 397468456690 state 0x0 ref 1 00000100:00100000:3.0:1789762874.179165:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397518456690 time 397468456690 state 0x1 ref 1 00000100:00100000:2.0:1789762874.214632:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397518456690 time 397518534625 state 0x0 ref 1 00000100:00100000:2.0:1789762874.214648:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015071360:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:2.0:1789762874.214683:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 49283072, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x17809 00010000:00010000:2.0:1789762874.214689:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d71 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.214701:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:2.0:1789762874.214705:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762874.214709:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.214720:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762874.225299:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 49283072 -> 50331648 00000001:00080000:3.0:1789762874.225303:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968079 is committed 00002000:00010000:2.0:1789762874.225304:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 96265 -> 98313 00010000:00010000:2.0:1789762874.225317:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d71 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.225327:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d71 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.225334:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.225339:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4de00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762874.225343:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d71 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.225348:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d71 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.225361:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d71 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.225366:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d71 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762874.225419:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015071360:12345-192.168.203.48@tcp:4:dd.0 Request processed in 10773us (46377us total) trans 4294968079 rc 0/0 00000100:00100000:2.0:1789762874.225441:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 973 00000100:00100000:2.0:1789762874.225450:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397518456690 time 397518534625 state 0x2 ref 0 00000100:00100000:0.0:1789762874.227622:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762874.227664:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015071488 00000100:00100000:2.0:1789762874.227671:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015071488 00000100:00100000:2.0:1789762874.227708:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397518456690 time 397518534625 state 0x0 ref 1 00000100:00100000:2.0:1789762874.227722:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397568534625 time 397518534625 state 0x1 ref 1 00000100:00100000:3.0:1789762874.264713:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397568534625 time 397568615139 state 0x0 ref 1 00000100:00100000:3.0:1789762874.264731:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015071488:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762874.264768:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 50331648, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x18009 00010000:00010000:3.0:1789762874.264777:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04d78 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.264791:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:3.0:1789762874.264797:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762874.264817:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04d78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.264832:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04d78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762874.276063:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968080 is committed 00002000:00010000:0.0:1789762874.276077:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 50331648 -> 51380224 00002000:00010000:0.0:1789762874.276084:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 98313 -> 100361 00010000:00010000:0.0:1789762874.276119:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04d78 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.276131:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04d78 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.276144:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04d78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.276150:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762874.276153:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04d78 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.276159:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04d78 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.276173:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04d78 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.276178:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04d78 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762874.276240:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015071488:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11511us (48617us total) trans 4294968080 rc 0/0 00000100:00100000:0.0:1789762874.276248:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 974 00000100:00100000:0.0:1789762874.276257:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397568534625 time 397568615139 state 0x2 ref 0 00000100:00100000:0.0:1789762874.278752:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.278794:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015071616 00000100:00100000:3.0:1789762874.278802:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015071616 00000100:00100000:3.0:1789762874.278847:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397568534625 time 397568615139 state 0x0 ref 1 00000100:00100000:3.0:1789762874.278868:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397618615139 time 397568615139 state 0x1 ref 1 00010000:00010000:2.0:1789762874.295202:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762874.314863:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397618615139 time 397618762998 state 0x0 ref 1 00000100:00100000:0.0:1789762874.314883:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015071616:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762874.314933:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 51380224, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x18809 00010000:00010000:0.0:1789762874.314940:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04d7f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.314951:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:0.0:1789762874.314955:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762874.314960:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04d7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.314970:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04d7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762874.326071:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968081 is committed 00002000:00010000:0.0:1789762874.326113:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 51380224 -> 52428800 00002000:00010000:0.0:1789762874.326119:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 100361 -> 102409 00010000:00010000:0.0:1789762874.326130:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04d7f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.326139:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04d7f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.326146:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04d7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.326151:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762874.326154:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04d7f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.326171:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04d7f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.326186:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04d7f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.326191:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04d7f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762874.326253:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015071616:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11374us (47498us total) trans 4294968081 rc 0/0 00000100:00100000:0.0:1789762874.326261:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 975 00000100:00100000:0.0:1789762874.326270:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397618615139 time 397618762998 state 0x2 ref 0 00000100:00100000:1.0:1789762874.328484:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.328532:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015071744 00000100:00100000:3.0:1789762874.328541:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015071744 00000100:00100000:3.0:1789762874.328584:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397618615139 time 397618762998 state 0x0 ref 1 00000100:00100000:3.0:1789762874.328600:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397668762998 time 397618762998 state 0x1 ref 1 00000100:00100000:2.0:1789762874.364940:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397668762998 time 397668838302 state 0x0 ref 1 00000100:00100000:2.0:1789762874.364962:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015071744:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:2.0:1789762874.364998:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 52428800, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x19009 00010000:00010000:2.0:1789762874.365004:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d86 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.365016:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:2.0:1789762874.365019:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762874.365024:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.365034:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762874.375937:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968082 is committed 00002000:00010000:0.0:1789762874.375956:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 52428800 -> 53477376 00002000:00010000:0.0:1789762874.375963:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 102409 -> 104457 00010000:00010000:0.0:1789762874.375975:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d86 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.375988:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d86 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.375995:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.376011:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4de00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762874.376015:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d86 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.376022:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d86 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.376035:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d86 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.376040:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000016074eb0/0x416193e012b04d86 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762874.376119:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015071744:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11162us (47634us total) trans 4294968082 rc 0/0 00000100:00100000:0.0:1789762874.376128:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 976 00000100:00100000:0.0:1789762874.376137:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397668762998 time 397668838302 state 0x2 ref 0 00000100:00100000:1.0:1789762874.378275:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762874.378354:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015071872 00000100:00100000:2.0:1789762874.378362:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015071872 00000100:00100000:2.0:1789762874.378395:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397668762998 time 397668838302 state 0x0 ref 1 00000100:00100000:2.0:1789762874.378406:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397718838302 time 397668838302 state 0x1 ref 1 00000100:00100000:0.0:1789762874.415019:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397718838302 time 397718921945 state 0x0 ref 1 00000100:00100000:0.0:1789762874.415034:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015071872:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762874.415065:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 53477376, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x19809 00010000:00010000:0.0:1789762874.415071:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d8d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.415115:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:0.0:1789762874.415119:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762874.415125:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.415136:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762874.425718:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968083 is committed 00002000:00010000:0.0:1789762874.425727:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:0.0:1789762874.425733:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 104457 -> 106505 00010000:00010000:0.0:1789762874.425742:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d8d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.425752:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d8d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.425758:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.425762:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762874.425766:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d8d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.425773:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d8d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.425797:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d8d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.425818:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d8d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762874.425874:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015071872:12345-192.168.203.48@tcp:4:dd.0 Request processed in 10842us (47599us total) trans 4294968083 rc 0/0 00000100:00100000:0.0:1789762874.425882:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 977 00000100:00100000:0.0:1789762874.425890:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397718838302 time 397718921945 state 0x2 ref 0 00000100:00100000:1.0:1789762874.427952:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.427994:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015072000 00000100:00100000:3.0:1789762874.428002:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015072000 00000100:00100000:3.0:1789762874.428035:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397718838302 time 397718921945 state 0x0 ref 1 00000100:00100000:3.0:1789762874.428046:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397768921945 time 397718921945 state 0x1 ref 1 00000100:00100000:0.0:1789762874.465116:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397768921945 time 397768995806 state 0x0 ref 1 00000100:00100000:0.0:1789762874.465137:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015072000:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762874.465184:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 54525952, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x1a009 00010000:00010000:0.0:1789762874.465190:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d94 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.465203:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:0.0:1789762874.465207:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762874.465212:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.465224:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762874.466309:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762874.466343:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762874.466368:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762874.466371:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015072128 00000100:00100000:2.0:1789762874.466380:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015072128 00000100:00100000:0.0:1789762874.466394:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015072384 00000100:00100000:0.0:1789762874.466399:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015072384 00000100:00100000:2.0:1789762874.466435:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 328 00000100:00100000:2.0:1789762874.466446:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015072128:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762874.466448:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015072256 00000100:00080000:1.0:1789762874.466452:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762874.466453:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015072256 00000100:00100000:1.0:1789762874.466482:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 308 00000100:00100000:0.0:1789762874.466490:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 309 00000100:00100000:1.0:1789762874.466495:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015072384:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762874.466502:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702015072256:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762874.466506:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015072128:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 61us (196us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762874.466513:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 328 00000100:00100000:1.0:1789762874.466538:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015072384:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 47us (193us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762874.466545:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702015072256:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 44us (176us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762874.466552:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 308 00000100:00100000:0.0:1789762874.466565:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 309 00000100:00080000:1.0:1789762874.466578:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:2.0:1789762874.475940:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968084 is committed 00002000:00010000:0.0:1789762874.475961:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 54525952 -> 55574528 00002000:00010000:0.0:1789762874.475967:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 106505 -> 108553 00010000:00010000:0.0:1789762874.475976:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d94 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.475985:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d94 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.475992:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.475997:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762874.476000:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d94 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.476014:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d94 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.476029:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d94 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.476034:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d94 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762874.476117:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015072000:12345-192.168.203.48@tcp:4:dd.0 Request processed in 10953us (48132us total) trans 4294968084 rc 0/0 00000100:00100000:0.0:1789762874.476127:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 978 00000100:00100000:0.0:1789762874.476135:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397768921945 time 397768995806 state 0x2 ref 0 00000100:00100000:1.0:1789762874.478376:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.478418:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015072512 00000100:00100000:3.0:1789762874.478424:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015072512 00000100:00100000:3.0:1789762874.478455:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397768921945 time 397768995806 state 0x0 ref 1 00000100:00100000:3.0:1789762874.478466:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397818995806 time 397768995806 state 0x1 ref 1 00000100:00100000:0.0:1789762874.487212:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022180096:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762874.487249:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762874.487272:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022180096 00000100:00100000:2.0:1789762874.487278:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022180096 00000100:00080000:1.0:1789762874.487314:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762874.487325:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:2.0:1789762874.487335:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022180096:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762874.487349:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526450688 pending 0 free 3821010944 avail 3802136576 00000100:00100000:2.0:1789762874.487389:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022180096:12345-0@lo:13:osp-pre-0-0.0 Request processed in 57us (139us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762874.487395:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000100:00100000:0.0:1789762874.487427:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022180096:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762874.515207:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397818995806 time 397819108293 state 0x0 ref 1 00000100:00100000:0.0:1789762874.515227:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015072512:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762874.515273:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 55574528, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x1a809 00010000:00010000:0.0:1789762874.515280:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d9b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.515302:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:0.0:1789762874.515306:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762874.515311:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.515321:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762874.526204:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 55574528 -> 56623104 00000001:00080000:3.0:1789762874.526204:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968085 is committed 00002000:00010000:2.0:1789762874.526211:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 108553 -> 110601 00010000:00010000:2.0:1789762874.526228:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d9b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.526237:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d9b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.526243:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.526248:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762874.526251:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d9b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.526258:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d9b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.526272:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d9b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.526277:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04d9b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762874.526337:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015072512:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11115us (47960us total) trans 4294968085 rc 0/0 00000100:00100000:2.0:1789762874.526345:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 979 00000100:00100000:2.0:1789762874.526355:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397818995806 time 397819108293 state 0x2 ref 0 00000100:00100000:1.0:1789762874.529549:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762874.529593:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015072640 00000100:00100000:2.0:1789762874.529603:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015072640 00000100:00100000:2.0:1789762874.529646:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397818995806 time 397819108293 state 0x0 ref 1 00000100:00100000:2.0:1789762874.529657:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397869108293 time 397819108293 state 0x1 ref 1 00000100:00100000:0.0:1789762874.565294:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397869108293 time 397869194401 state 0x0 ref 1 00000100:00100000:0.0:1789762874.565313:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015072640:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762874.565364:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 56623104, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x1b009 00010000:00010000:0.0:1789762874.565374:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.565389:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:0.0:1789762874.565393:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762874.565397:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.565409:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762874.576690:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968086 is committed 00002000:00010000:0.0:1789762874.576707:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:0.0:1789762874.576712:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 110601 -> 112649 00010000:00010000:0.0:1789762874.576728:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.576737:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.576744:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.576749:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762874.576752:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.576759:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.576773:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.576777:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762874.576845:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015072640:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11536us (47299us total) trans 4294968086 rc 0/0 00000100:00100000:0.0:1789762874.576853:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 980 00000100:00100000:0.0:1789762874.576861:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397869108293 time 397869194401 state 0x2 ref 0 00000100:00100000:1.0:1789762874.579080:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.579131:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015072768 00000100:00100000:3.0:1789762874.579138:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015072768 00000100:00100000:3.0:1789762874.579172:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397869108293 time 397869194401 state 0x0 ref 1 00000100:00100000:3.0:1789762874.579183:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397919194401 time 397869194401 state 0x1 ref 1 00000100:00100000:0.0:1789762874.615379:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397919194401 time 397919278503 state 0x0 ref 1 00000100:00100000:0.0:1789762874.615399:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015072768:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762874.615441:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 57671680, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x1b809 00010000:00010000:0.0:1789762874.615448:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.615461:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:0.0:1789762874.615466:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762874.615473:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.615488:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762874.626079:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968087 is committed 00002000:00010000:0.0:1789762874.626131:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:0.0:1789762874.626136:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 112649 -> 114697 00010000:00010000:0.0:1789762874.626148:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.626157:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.626164:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.626181:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762874.626184:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.626191:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.626205:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.626210:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04da9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762874.626277:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015072768:12345-192.168.203.48@tcp:4:dd.0 Request processed in 10882us (47196us total) trans 4294968087 rc 0/0 00000100:00100000:0.0:1789762874.626286:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 981 00000100:00100000:0.0:1789762874.626296:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397919194401 time 397919278503 state 0x2 ref 0 00000100:00100000:0.0:1789762874.628652:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.628691:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015072896 00000100:00100000:3.0:1789762874.628700:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015072896 00000100:00100000:3.0:1789762874.628737:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397919194401 time 397919278503 state 0x0 ref 1 00000100:00100000:3.0:1789762874.628749:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397969278503 time 397919278503 state 0x1 ref 1 00000100:00100000:2.0:1789762874.665477:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397969278503 time 397969378127 state 0x0 ref 1 00000100:00100000:2.0:1789762874.665494:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015072896:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:2.0:1789762874.665530:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 58720256, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x1c009 00010000:00010000:2.0:1789762874.665537:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e49ca825/0x416193e012b04db0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.665549:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=397s v=(0 0) 00010000:00010000:2.0:1789762874.665553:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762874.665558:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e49ca825/0x416193e012b04db0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762874.665567:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e49ca825/0x416193e012b04db0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762874.675676:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968088 is committed 00002000:00010000:0.0:1789762874.675720:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:0.0:1789762874.675725:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 114697 -> 116745 00010000:00010000:0.0:1789762874.675734:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e49ca825/0x416193e012b04db0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.675744:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e49ca825/0x416193e012b04db0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.675758:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e49ca825/0x416193e012b04db0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.675763:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762874.675766:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e49ca825/0x416193e012b04db0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.675773:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e49ca825/0x416193e012b04db0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.675785:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e49ca825/0x416193e012b04db0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.675791:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e49ca825/0x416193e012b04db0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762874.675867:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015072896:12345-192.168.203.48@tcp:4:dd.0 Request processed in 10375us (47213us total) trans 4294968088 rc 0/0 00000100:00100000:0.0:1789762874.675878:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 982 00000100:00100000:0.0:1789762874.675890:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397969278503 time 397969378127 state 0x2 ref 0 00000100:00100000:1.0:1789762874.678146:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762874.678199:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015073024 00000100:00100000:2.0:1789762874.678207:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015073024 00000100:00100000:2.0:1789762874.678253:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 397969278503 time 397969378127 state 0x0 ref 1 00000100:00100000:2.0:1789762874.678268:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398019378127 time 397969378127 state 0x1 ref 1 00000100:00100000:0.0:1789762874.715571:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398019378127 time 398019470677 state 0x0 ref 1 00000100:00100000:0.0:1789762874.715590:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015073024:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762874.715643:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 59768832, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x1c809 00010000:00010000:0.0:1789762874.715649:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04db7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.715661:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:0.0:1789762874.715664:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762874.715669:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04db7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.715681:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04db7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762874.727268:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 59768832 -> 60817408 00000001:00080000:2.0:1789762874.727269:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968089 is committed 00002000:00010000:3.0:1789762874.727275:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 116745 -> 118793 00010000:00010000:3.0:1789762874.727286:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04db7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.727296:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04db7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.727303:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04db7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.727309:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86185200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762874.727312:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04db7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.727319:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04db7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.727332:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04db7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.727337:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006a984f90/0x416193e012b04db7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762874.727398:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015073024:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11812us (49250us total) trans 4294968089 rc 0/0 00000100:00100000:3.0:1789762874.727406:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 983 00000100:00100000:3.0:1789762874.727415:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398019378127 time 398019470677 state 0x2 ref 0 00000100:00100000:1.0:1789762874.729596:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.729637:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015073152 00000100:00100000:3.0:1789762874.729647:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015073152 00000100:00100000:3.0:1789762874.729688:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398019378127 time 398019470677 state 0x0 ref 1 00000100:00100000:3.0:1789762874.729700:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398069470677 time 398019470677 state 0x1 ref 1 00000100:00100000:0.0:1789762874.765694:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398069470677 time 398069595298 state 0x0 ref 1 00000100:00100000:0.0:1789762874.765713:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015073152:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762874.765753:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 60817408, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x1d009 00010000:00010000:0.0:1789762874.765760:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dbe lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.765771:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:0.0:1789762874.765775:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762874.765779:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.765790:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762874.778006:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968090 is committed 00002000:00010000:0.0:1789762874.778016:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 60817408 -> 61865984 00002000:00010000:0.0:1789762874.778021:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 118793 -> 120841 00010000:00010000:0.0:1789762874.778032:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dbe lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.778042:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dbe lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.778048:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.778053:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762874.778056:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dbe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.778063:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dbe lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.778078:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dbe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.778083:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dbe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762874.778162:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015073152:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12451us (48564us total) trans 4294968090 rc 0/0 00000100:00100000:0.0:1789762874.778177:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 984 00000100:00100000:0.0:1789762874.778186:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398069470677 time 398069595298 state 0x2 ref 0 00000100:00100000:0.0:1789762874.780541:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.780598:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015073280 00000100:00100000:3.0:1789762874.780607:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015073280 00000100:00100000:3.0:1789762874.780662:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398069470677 time 398069595298 state 0x0 ref 1 00000100:00100000:3.0:1789762874.780678:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398119595298 time 398069595298 state 0x1 ref 1 00000100:00100000:0.0:1789762874.815824:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398119595298 time 398119708607 state 0x0 ref 1 00000100:00100000:0.0:1789762874.815853:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015073280:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762874.815899:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 61865984, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x1d809 00010000:00010000:0.0:1789762874.815906:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dc5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.815918:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:0.0:1789762874.815922:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762874.815927:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.815938:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762874.828150:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968091 is committed 00002000:00010000:1.0:1789762874.828200:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 61865984 -> 62914560 00002000:00010000:1.0:1789762874.828207:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 120841 -> 122889 00010000:00010000:1.0:1789762874.828222:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dc5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.828254:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dc5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.828264:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.828273:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762874.828279:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dc5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.828287:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dc5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.828304:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dc5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.828310:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cb3085bc/0x416193e012b04dc5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762874.828401:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015073280:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12550us (47855us total) trans 4294968091 rc 0/0 00000100:00100000:1.0:1789762874.828521:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 985 00000100:00100000:1.0:1789762874.828532:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398119595298 time 398119708607 state 0x2 ref 0 00000100:00100000:1.0:1789762874.831010:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762874.831055:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015073408 00000100:00100000:2.0:1789762874.831064:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015073408 00000100:00100000:2.0:1789762874.831139:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398119595298 time 398119708607 state 0x0 ref 1 00000100:00100000:2.0:1789762874.831156:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398169708607 time 398119708607 state 0x1 ref 1 00000100:00100000:3.0:1789762874.865917:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398169708607 time 398169815568 state 0x0 ref 1 00000100:00100000:3.0:1789762874.865937:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015073408:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762874.865981:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 62914560, mtime: 0x6aad9d3a, atime: 0x0, ctime: 0x6aad9d3a, blocks: 0x1e009 00010000:00010000:3.0:1789762874.865989:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04dcc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.866006:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:3.0:1789762874.866010:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762874.866014:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04dcc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762874.866025:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04dcc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762874.877772:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968092 is committed 00002000:00010000:1.0:1789762874.877796:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:1.0:1789762874.877813:0:14914:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb mtime from disk: 1789762874 -> 1789762875 00002000:00010000:1.0:1789762874.877817:0:14914:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb ctime from disk: 1789762874 -> 1789762875 00002000:00010000:1.0:1789762874.877820:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 122889 -> 124937 00010000:00010000:1.0:1789762874.877831:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04dcc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.877840:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04dcc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.877847:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04dcc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.877852:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762874.877855:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04dcc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.877862:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04dcc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.877876:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04dcc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.877881:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04dcc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762874.877945:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015073408:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12012us (46934us total) trans 4294968092 rc 0/0 00000100:00100000:1.0:1789762874.877954:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 986 00000100:00100000:1.0:1789762874.877964:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398169708607 time 398169815568 state 0x2 ref 0 00000100:00100000:2.0:1789762874.880181:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.880223:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015073536 00000100:00100000:3.0:1789762874.880230:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015073536 00000100:00100000:3.0:1789762874.880267:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398169708607 time 398169815568 state 0x0 ref 1 00000100:00100000:3.0:1789762874.880278:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398219815568 time 398169815568 state 0x1 ref 1 00000100:00100000:0.0:1789762874.916023:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398219815568 time 398219920703 state 0x0 ref 1 00000100:00100000:0.0:1789762874.916047:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015073536:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:0.0:1789762874.916115:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 63963136, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x1e809 00010000:00010000:0.0:1789762874.916136:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04dd3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.916148:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:0.0:1789762874.916152:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762874.916157:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04dd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762874.916168:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04dd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762874.927726:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968093 is committed 00002000:00010000:1.0:1789762874.927736:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 63963136 -> 65011712 00002000:00010000:1.0:1789762874.927742:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 124937 -> 126985 00010000:00010000:1.0:1789762874.927752:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04dd3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.927763:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04dd3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.927770:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04dd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.927775:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762874.927779:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04dd3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.927785:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04dd3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.927816:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04dd3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.927822:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000012ef762a/0x416193e012b04dd3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762874.927887:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015073536:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11848us (47705us total) trans 4294968093 rc 0/0 00000100:00100000:1.0:1789762874.927895:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 987 00000100:00100000:1.0:1789762874.927906:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398219815568 time 398219920703 state 0x2 ref 0 00000100:00100000:1.0:1789762874.930165:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762874.930203:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015073664 00000100:00100000:3.0:1789762874.930212:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015073664 00000100:00100000:3.0:1789762874.930249:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398219815568 time 398219920703 state 0x0 ref 1 00000100:00100000:3.0:1789762874.930261:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398269920703 time 398219920703 state 0x1 ref 1 00000100:00100000:1.0:1789762874.966143:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398269920703 time 398270037671 state 0x0 ref 1 00000100:00100000:1.0:1789762874.966169:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015073664:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762874.966234:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 65011712, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x1f009 00010000:00010000:1.0:1789762874.966241:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04dda lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.966255:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762874.966258:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762874.966263:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04dda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.966274:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04dda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762874.978510:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968094 is committed 00002000:00010000:1.0:1789762874.978534:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 65011712 -> 66060288 00002000:00010000:1.0:1789762874.978541:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 126985 -> 129033 00010000:00010000:1.0:1789762874.978555:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04dda lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.978578:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04dda lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.978587:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04dda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.978595:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762874.978600:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04dda lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.978609:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04dda lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.978628:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04dda lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762874.978637:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04dda lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762874.978736:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015073664:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12573us (48567us total) trans 4294968094 rc 0/0 00000100:00100000:1.0:1789762874.978747:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 988 00000100:00100000:1.0:1789762874.978757:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398269920703 time 398270037671 state 0x2 ref 0 00000100:00100000:2.0:1789762874.981418:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762874.981465:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015073792 00000100:00100000:1.0:1789762874.981476:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015073792 00000100:00100000:1.0:1789762874.981526:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398269920703 time 398270037671 state 0x0 ref 1 00000100:00100000:1.0:1789762874.981542:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398320037671 time 398270037671 state 0x1 ref 1 00000100:00100000:1.0:1789762874.999284:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8a6b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022180224:0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.999284:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8a6b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022180480:0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.999294:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e879c4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022180352:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762874.999323:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8a6b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022180608:0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.999369:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762874.999378:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762874.999379:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762874.999395:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022180224 00000100:00100000:0.0:1789762874.999401:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022180224 00000100:00100000:1.0:1789762874.999432:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8a6b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022180736:0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.999437:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8a6b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022180864:0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.999438:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8a6b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022180992:0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.999463:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 329 00000100:00100000:0.0:1789762874.999475:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850df450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022180224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.999478:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762874.999508:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762874.999522:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762874.999532:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850df450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022180224:12345-0@lo:400:kworker.0 Request processed in 59us (162us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762874.999535:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022180736 00000100:00100000:0.0:1789762874.999540:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 329 00000100:00100000:1.0:1789762874.999543:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022180736 00000100:00100000:3.0:1789762874.999578:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022180352 00000100:00100000:2.0:1789762874.999579:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022180480 00000100:00100000:1.0:1789762874.999585:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022180992 00000100:00100000:3.0:1789762874.999585:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022180352 00000100:00100000:0.0:1789762874.999586:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762874.999586:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022180480 00000100:00100000:1.0:1789762874.999592:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022180992 00000100:00100000:1.0:1789762874.999630:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:1.0:1789762874.999641:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022180736:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.999643:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022180864 00000100:00100000:2.0:1789762874.999647:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022180608 00000100:00100000:3.0:1789762874.999648:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:0.0:1789762874.999650:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022180864 00000100:00100000:2.0:1789762874.999657:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022180608 00000100:00100000:3.0:1789762874.999662:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022180352:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762874.999685:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022180736:12345-0@lo:400:kworker.0 Request processed in 47us (209us total) trans 0 rc 0/0 00002000:00000024:3.0:1789762874.999690:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762874.999693:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 310 00000100:00100000:1.0:1789762874.999695:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:0.0:1789762874.999709:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022180480:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.999711:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 311 00000100:00100000:1.0:1789762874.999717:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 156 00000100:00100000:2.0:1789762874.999730:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022180864:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762874.999731:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022180992:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.999741:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022180352:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (366us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762874.999748:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00100000:0.0:1789762874.999768:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022180480:12345-0@lo:400:kworker.0 Request processed in 63us (395us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762874.999775:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:0.0:1789762874.999783:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 310 00000100:00100000:1.0:1789762874.999784:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022180992:12345-0@lo:400:kworker.0 Request processed in 56us (275us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762874.999788:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022180608:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.999815:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022180864:12345-0@lo:400:kworker.0 Request processed in 75us (278us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762874.999842:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 311 00000100:00100000:0.0:1789762874.999866:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8a6b5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022180992:0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.999871:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022180608:12345-0@lo:400:kworker.0 Request processed in 84us (282us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762874.999883:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8a6b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022180480:0@lo:400:kworker.0 00000100:00100000:3.0:1789762874.999884:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00100000:1.0:1789762874.999899:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 156 00000100:00080000:2.0:1789762874.999899:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762874.999918:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762874.999940:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8a6b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022180608:0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.999945:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8a6b7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022180736:0@lo:400:kworker.0 00000100:00100000:0.0:1789762874.999963:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8a6b4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022180224:0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.999968:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8a6b4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022180864:0@lo:400:kworker.0 00000100:00100000:2.0:1789762874.999995:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e879c4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022180352:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762875.016203:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398320037671 time 398320101532 state 0x0 ref 1 00000100:00100000:1.0:1789762875.016236:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015073792:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.016283:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 66060288, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x1f809 00010000:00010000:1.0:1789762875.016293:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762875.016305:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762875.016311:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762875.016318:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.016326:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.016339:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762875.029616:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968095 is committed 00002000:00010000:1.0:1789762875.029639:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 66060288 -> 67108864 00002000:00010000:1.0:1789762875.029645:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 129033 -> 131081 00010000:00010000:1.0:1789762875.029655:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.029666:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.029672:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.029677:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.029680:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.029692:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.029707:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.029713:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.029782:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015073792:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13561us (48361us total) trans 4294968095 rc 0/0 00000100:00100000:1.0:1789762875.029791:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 989 00000100:00100000:1.0:1789762875.029815:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398320037671 time 398320101532 state 0x2 ref 0 00000100:00100000:0.0:1789762875.032150:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.032194:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015073920 00000100:00100000:3.0:1789762875.032202:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015073920 00000100:00100000:3.0:1789762875.032240:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398320037671 time 398320101532 state 0x0 ref 1 00000100:00100000:3.0:1789762875.032252:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398370101532 time 398320101532 state 0x1 ref 1 00000100:00100000:1.0:1789762875.066333:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398370101532 time 398370229343 state 0x0 ref 1 00000100:00100000:1.0:1789762875.066360:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015073920:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.066413:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 67108864, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x20009 00010000:00010000:1.0:1789762875.066420:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.066433:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762875.066437:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.066441:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.066453:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762875.078232:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968096 is committed 00002000:00010000:1.0:1789762875.078271:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 67108864 -> 68157440 00002000:00010000:1.0:1789762875.078277:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 131081 -> 133129 00010000:00010000:1.0:1789762875.078292:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.078307:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.078318:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.078326:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.078332:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.078342:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.078362:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.078371:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04de8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.078460:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015073920:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12107us (46306us total) trans 4294968096 rc 0/0 00000100:00100000:1.0:1789762875.078471:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 990 00000100:00100000:1.0:1789762875.078486:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398370101532 time 398370229343 state 0x2 ref 0 00000100:00100000:2.0:1789762875.080923:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.080973:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015074048 00000100:00100000:3.0:1789762875.080983:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015074048 00000100:00100000:3.0:1789762875.081030:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398370101532 time 398370229343 state 0x0 ref 1 00000100:00100000:3.0:1789762875.081050:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398420229343 time 398370229343 state 0x1 ref 1 00000100:00100000:1.0:1789762875.116486:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398420229343 time 398420378372 state 0x0 ref 1 00000100:00100000:1.0:1789762875.116522:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015074048:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.116575:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 68157440, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x20809 00010000:00010000:1.0:1789762875.116602:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04def lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.116617:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762875.116620:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.116625:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04def lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.116636:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04def lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762875.128916:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 68157440 -> 69206016 00000001:00080000:3.0:1789762875.128919:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968097 is committed 00002000:00010000:2.0:1789762875.128923:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 133129 -> 135177 00010000:00010000:2.0:1789762875.128935:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04def lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.128945:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04def lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.128952:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04def lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.128956:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762875.128961:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04def lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.128968:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04def lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.128992:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04def lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.128998:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b04def lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762875.129065:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015074048:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12559us (48141us total) trans 4294968097 rc 0/0 00000100:00100000:2.0:1789762875.129073:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 991 00000100:00100000:2.0:1789762875.129082:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398420229343 time 398420378372 state 0x2 ref 0 00000100:00100000:0.0:1789762875.131604:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762875.131652:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015074176 00000100:00100000:1.0:1789762875.131661:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015074176 00000100:00100000:1.0:1789762875.131699:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398420229343 time 398420378372 state 0x0 ref 1 00000100:00100000:1.0:1789762875.131713:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398470378372 time 398420378372 state 0x1 ref 1 00000100:00100000:1.0:1789762875.166559:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398470378372 time 398470454903 state 0x0 ref 1 00000100:00100000:1.0:1789762875.166596:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015074176:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.166647:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 69206016, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x21009 00010000:00010000:1.0:1789762875.166655:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04df6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.166668:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762875.166672:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.166677:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04df6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.166687:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04df6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762875.177987:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968098 is committed 00002000:00010000:3.0:1789762875.177991:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 69206016 -> 70254592 00002000:00010000:3.0:1789762875.177997:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 135177 -> 137225 00010000:00010000:3.0:1789762875.178008:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04df6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762875.178019:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04df6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762875.178026:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04df6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762875.178031:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762875.178034:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04df6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762875.178041:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04df6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762875.178056:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04df6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762875.178061:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b04df6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762875.178162:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015074176:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11584us (46556us total) trans 4294968098 rc 0/0 00000100:00100000:3.0:1789762875.178170:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 992 00000100:00100000:3.0:1789762875.178181:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398470378372 time 398470454903 state 0x2 ref 0 00000100:00100000:1.0:1789762875.180853:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.180890:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015074304 00000100:00100000:3.0:1789762875.180900:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015074304 00000100:00100000:3.0:1789762875.180939:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398470378372 time 398470454903 state 0x0 ref 1 00000100:00100000:3.0:1789762875.180950:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398520454903 time 398470454903 state 0x1 ref 1 00000100:00100000:1.0:1789762875.216689:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398520454903 time 398520578069 state 0x0 ref 1 00000100:00100000:1.0:1789762875.216722:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b39180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015074304:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.216782:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 70254592, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x21809 00010000:00010000:1.0:1789762875.216792:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04dfd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.216822:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762875.216828:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.216834:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04dfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.216849:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04dfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762875.228365:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968099 is committed 00002000:00010000:1.0:1789762875.228387:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 70254592 -> 71303168 00002000:00010000:1.0:1789762875.228393:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 137225 -> 139273 00010000:00010000:1.0:1789762875.228406:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04dfd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.228416:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04dfd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.228423:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04dfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.228428:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.228431:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04dfd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.228438:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04dfd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.228461:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04dfd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.228470:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b04dfd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.228559:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b39180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015074304:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11849us (47706us total) trans 4294968099 rc 0/0 00000100:00100000:1.0:1789762875.228568:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 993 00000100:00100000:1.0:1789762875.228578:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398520454903 time 398520578069 state 0x2 ref 0 00000100:00100000:0.0:1789762875.231125:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.231168:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015074432 00000100:00100000:3.0:1789762875.231176:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015074432 00000100:00100000:3.0:1789762875.231217:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398520454903 time 398520578069 state 0x0 ref 1 00000100:00100000:3.0:1789762875.231228:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398570578069 time 398520578069 state 0x1 ref 1 00000100:00100000:2.0:1789762875.266766:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398570578069 time 398570663516 state 0x0 ref 1 00000100:00100000:2.0:1789762875.266791:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015074432:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:2.0:1789762875.266855:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 71303168, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x22009 00010000:00010000:2.0:1789762875.266863:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04e04 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.266876:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:2.0:1789762875.266879:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762875.266884:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04e04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.266912:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04e04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762875.279129:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968100 is committed 00002000:00010000:1.0:1789762875.279143:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 71303168 -> 72351744 00002000:00010000:1.0:1789762875.279148:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 139273 -> 141321 00010000:00010000:1.0:1789762875.279160:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04e04 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.279171:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04e04 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.279178:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04e04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.279183:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.279186:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04e04 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.279197:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04e04 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.279213:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04e04 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.279218:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b04e04 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.279285:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015074432:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12502us (48159us total) trans 4294968100 rc 0/0 00000100:00100000:1.0:1789762875.279293:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 994 00000100:00100000:1.0:1789762875.279303:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398570578069 time 398570663516 state 0x2 ref 0 00000100:00100000:2.0:1789762875.281653:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762875.281690:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015074560 00000100:00100000:1.0:1789762875.281698:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015074560 00000100:00100000:1.0:1789762875.281734:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398570578069 time 398570663516 state 0x0 ref 1 00000100:00100000:1.0:1789762875.281745:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398620663516 time 398570663516 state 0x1 ref 1 00000100:00100000:3.0:1789762875.316905:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398620663516 time 398620799724 state 0x0 ref 1 00000100:00100000:3.0:1789762875.316933:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015074560:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762875.316983:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 72351744, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x22809 00010000:00010000:3.0:1789762875.316991:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04e0b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762875.317004:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:3.0:1789762875.317008:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762875.317013:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04e0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762875.317023:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04e0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762875.319162:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789762875.328885:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968101 is committed 00002000:00010000:1.0:1789762875.328898:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 72351744 -> 73400320 00002000:00010000:1.0:1789762875.328904:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 141321 -> 143369 00010000:00010000:1.0:1789762875.328917:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04e0b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.328932:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04e0b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.328939:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04e0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.328944:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.328948:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04e0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.328954:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04e0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.328969:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04e0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.328975:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04e0b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.329049:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015074560:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12125us (47394us total) trans 4294968101 rc 0/0 00000100:00100000:1.0:1789762875.329057:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 995 00000100:00100000:1.0:1789762875.329067:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398620663516 time 398620799724 state 0x2 ref 0 00000100:00100000:1.0:1789762875.331537:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.331585:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015074688 00000100:00100000:3.0:1789762875.331594:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015074688 00000100:00100000:3.0:1789762875.331642:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398620663516 time 398620799724 state 0x0 ref 1 00000100:00100000:3.0:1789762875.331658:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398670799724 time 398620799724 state 0x1 ref 1 00000100:00100000:1.0:1789762875.366999:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398670799724 time 398670895145 state 0x0 ref 1 00000100:00100000:1.0:1789762875.367027:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015074688:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.367080:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 73400320, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x23009 00010000:00010000:1.0:1789762875.367110:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e12 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.367124:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762875.367128:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.367133:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.367144:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762875.378292:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968102 is committed 00002000:00010000:1.0:1789762875.378363:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 73400320 -> 74448896 00002000:00010000:1.0:1789762875.378368:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 143369 -> 145417 00010000:00010000:1.0:1789762875.378380:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e12 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.378389:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e12 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.378396:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.378405:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.378409:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e12 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.378415:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e12 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.378430:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e12 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.378435:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e12 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.378507:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015074688:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11490us (46968us total) trans 4294968102 rc 0/0 00000100:00100000:1.0:1789762875.378516:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 996 00000100:00100000:1.0:1789762875.378526:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398670799724 time 398670895145 state 0x2 ref 0 00000100:00100000:0.0:1789762875.380985:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.381026:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015074816 00000100:00100000:3.0:1789762875.381035:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015074816 00000100:00100000:3.0:1789762875.381074:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398670799724 time 398670895145 state 0x0 ref 1 00000100:00100000:3.0:1789762875.381100:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398720895145 time 398670895145 state 0x1 ref 1 00000100:00100000:1.0:1789762875.417139:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398720895145 time 398720995519 state 0x0 ref 1 00000100:00100000:1.0:1789762875.417166:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015074816:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.417214:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 74448896, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x23809 00010000:00010000:1.0:1789762875.417222:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e19 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.417235:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762875.417239:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.417245:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.417260:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762875.429253:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968103 is committed 00002000:00010000:2.0:1789762875.429275:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:2.0:1789762875.429280:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 145417 -> 147465 00010000:00010000:2.0:1789762875.429292:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e19 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.429302:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e19 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.429308:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.429314:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762875.429317:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e19 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.429324:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e19 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.429341:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e19 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.429356:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04e19 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762875.429441:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015074816:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12281us (48451us total) trans 4294968103 rc 0/0 00000100:00100000:2.0:1789762875.429452:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 997 00000100:00100000:2.0:1789762875.429467:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398720895145 time 398720995519 state 0x2 ref 0 00000100:00100000:0.0:1789762875.431980:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762875.432026:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015074944 00000100:00100000:1.0:1789762875.432035:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015074944 00000100:00100000:1.0:1789762875.432077:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398720895145 time 398720995519 state 0x0 ref 1 00000100:00100000:1.0:1789762875.432116:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398770995519 time 398720995519 state 0x1 ref 1 00000100:00100000:1.0:1789762875.467167:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398770995519 time 398771064265 state 0x0 ref 1 00000100:00100000:1.0:1789762875.467192:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015074944:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.467242:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 75497472, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x24009 00010000:00010000:1.0:1789762875.467250:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e20 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.467263:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762875.467266:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.467271:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.467283:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762875.479612:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968104 is committed 00002000:00010000:1.0:1789762875.479633:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:1.0:1789762875.479639:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 147465 -> 149513 00010000:00010000:1.0:1789762875.479668:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e20 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.479680:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e20 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.479687:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.479693:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.479697:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.479705:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e20 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.479721:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.479727:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.479823:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015074944:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12636us (47839us total) trans 4294968104 rc 0/0 00000100:00100000:1.0:1789762875.479835:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 998 00000100:00100000:1.0:1789762875.479862:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398770995519 time 398771064265 state 0x2 ref 0 00000100:00100000:0.0:1789762875.482944:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762875.482987:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015075072 00000100:00100000:1.0:1789762875.482997:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015075072 00000100:00100000:1.0:1789762875.483046:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398770995519 time 398771064265 state 0x0 ref 1 00000100:00100000:1.0:1789762875.483059:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398821064265 time 398771064265 state 0x1 ref 1 00000100:00100000:1.0:1789762875.517257:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398821064265 time 398821152776 state 0x0 ref 1 00000100:00100000:1.0:1789762875.517305:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015075072:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.517361:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 76546048, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x24809 00010000:00010000:1.0:1789762875.517369:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e27 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.517383:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762875.517387:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.517391:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.517410:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762875.528793:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968105 is committed 00002000:00010000:1.0:1789762875.528827:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:1.0:1789762875.528834:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 149513 -> 151561 00010000:00010000:1.0:1789762875.528847:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e27 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.528863:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e27 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.528873:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.528880:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.528883:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e27 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.528890:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e27 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.528905:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e27 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.528910:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000982b98dc/0x416193e012b04e27 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.528982:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015075072:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11705us (46044us total) trans 4294968105 rc 0/0 00000100:00100000:1.0:1789762875.528991:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 999 00000100:00100000:1.0:1789762875.529001:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398821064265 time 398821152776 state 0x2 ref 0 00000100:00100000:0.0:1789762875.531382:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.531425:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015075200 00000100:00100000:3.0:1789762875.531433:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015075200 00000100:00100000:3.0:1789762875.531471:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398821064265 time 398821152776 state 0x0 ref 1 00000100:00100000:3.0:1789762875.531483:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398871152776 time 398821152776 state 0x1 ref 1 00000100:00100000:2.0:1789762875.567324:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398871152776 time 398871220955 state 0x0 ref 1 00000100:00100000:2.0:1789762875.567354:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015075200:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:2.0:1789762875.567404:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 77594624, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x25009 00010000:00010000:2.0:1789762875.567411:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04e2e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.567425:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:2.0:1789762875.567429:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762875.567434:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04e2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762875.567445:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04e2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762875.578511:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968106 is committed 00002000:00010000:1.0:1789762875.578514:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:1.0:1789762875.578521:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 151561 -> 153609 00010000:00010000:1.0:1789762875.578535:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04e2e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.578549:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04e2e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.578560:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04e2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.578568:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4c600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.578573:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04e2e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.578579:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04e2e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.578594:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04e2e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.578599:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008cdb5dbf/0x416193e012b04e2e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.578671:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015075200:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11329us (47287us total) trans 4294968106 rc 0/0 00000100:00100000:1.0:1789762875.578679:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1000 00000100:00100000:1.0:1789762875.578689:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398871152776 time 398871220955 state 0x2 ref 0 00000100:00100000:1.0:1789762875.580979:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762875.581026:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015075328 00000100:00100000:2.0:1789762875.581041:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015075328 00000100:00100000:2.0:1789762875.581106:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398871152776 time 398871220955 state 0x0 ref 1 00000100:00100000:2.0:1789762875.581123:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398921220955 time 398871220955 state 0x1 ref 1 00000100:00100000:1.0:1789762875.617440:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398921220955 time 398921336812 state 0x0 ref 1 00000100:00100000:1.0:1789762875.617464:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b38e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015075328:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.617515:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 78643200, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x25809 00010000:00010000:1.0:1789762875.617523:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e35 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.617535:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762875.617539:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.617543:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.617554:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762875.629656:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968107 is committed 00002000:00010000:1.0:1789762875.629673:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:1.0:1789762875.629679:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 153609 -> 155657 00010000:00010000:1.0:1789762875.629690:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e35 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.629701:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e35 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.629708:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.629713:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.629717:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e35 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.629723:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e35 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.629738:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e35 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.629742:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e35 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.629819:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b38e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015075328:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12363us (48838us total) trans 4294968107 rc 0/0 00000100:00100000:1.0:1789762875.629826:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1001 00000100:00100000:1.0:1789762875.629836:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398921220955 time 398921336812 state 0x2 ref 0 00000100:00100000:1.0:1789762875.631999:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.632041:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015075456 00000100:00100000:3.0:1789762875.632050:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015075456 00000100:00100000:3.0:1789762875.632105:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398921220955 time 398921336812 state 0x0 ref 1 00000100:00100000:3.0:1789762875.632118:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398971336812 time 398921336812 state 0x1 ref 1 00000100:00100000:1.0:1789762875.667561:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398971336812 time 398971452970 state 0x0 ref 1 00000100:00100000:1.0:1789762875.667589:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b38a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015075456:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.667638:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 79691776, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x26009 00010000:00010000:1.0:1789762875.667645:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e3c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.667658:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=398s v=(0 0) 00010000:00010000:1.0:1789762875.667661:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.667666:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.667676:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762875.679382:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968108 is committed 00002000:00010000:1.0:1789762875.679454:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:1.0:1789762875.679459:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 155657 -> 157705 00010000:00010000:1.0:1789762875.679470:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e3c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.679480:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e3c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.679487:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.679497:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.679501:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e3c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.679507:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e3c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.679522:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e3c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.679527:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e3c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.679595:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b38a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015075456:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12014us (47594us total) trans 4294968108 rc 0/0 00000100:00100000:1.0:1789762875.679604:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1002 00000100:00100000:1.0:1789762875.679615:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398971336812 time 398971452970 state 0x2 ref 0 00000100:00100000:1.0:1789762875.682024:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.682066:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015075584 00000100:00100000:3.0:1789762875.682076:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015075584 00000100:00100000:3.0:1789762875.682134:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 398971336812 time 398971452970 state 0x0 ref 1 00000100:00100000:3.0:1789762875.682145:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399021452970 time 398971452970 state 0x1 ref 1 00000100:00100000:1.0:1789762875.717651:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399021452970 time 399021546927 state 0x0 ref 1 00000100:00100000:1.0:1789762875.717679:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b38000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015075584:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.717730:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 80740352, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x26809 00010000:00010000:1.0:1789762875.717740:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e43 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.717753:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762875.717757:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.717761:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.717774:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762875.730333:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968109 is committed 00002000:00010000:1.0:1789762875.730395:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 80740352 -> 81788928 00002000:00010000:1.0:1789762875.730400:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 157705 -> 159753 00010000:00010000:1.0:1789762875.730410:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e43 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.730421:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e43 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.730427:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.730432:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.730436:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e43 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.730443:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e43 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.730458:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e43 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.730464:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e43 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.730527:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b38000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015075584:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12859us (48501us total) trans 4294968109 rc 0/0 00000100:00100000:1.0:1789762875.730536:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1003 00000100:00100000:1.0:1789762875.730546:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399021452970 time 399021546927 state 0x2 ref 0 00000100:00100000:0.0:1789762875.733143:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762875.733223:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015075712 00000100:00100000:2.0:1789762875.733232:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015075712 00000100:00100000:2.0:1789762875.733271:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399021452970 time 399021546927 state 0x0 ref 1 00000100:00100000:2.0:1789762875.733282:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399071546927 time 399021546927 state 0x1 ref 1 00000100:00100000:1.0:1789762875.767746:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399071546927 time 399071644000 state 0x0 ref 1 00000100:00100000:1.0:1789762875.767769:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015075712:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.767848:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 81788928, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x27009 00010000:00010000:1.0:1789762875.767856:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e4a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.767870:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762875.767873:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.767878:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.767889:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762875.780208:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:1.0:1789762875.780215:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 159753 -> 161801 00000001:00080000:2.0:1789762875.780215:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968110 is committed 00010000:00010000:1.0:1789762875.780230:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e4a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.780241:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e4a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.780249:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.780254:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.780257:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e4a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.780264:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e4a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.780279:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e4a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.780284:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e4a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.780379:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015075712:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12613us (47231us total) trans 4294968110 rc 0/0 00000100:00100000:1.0:1789762875.780390:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1004 00000100:00100000:1.0:1789762875.780417:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399071546927 time 399071644000 state 0x2 ref 0 00000100:00100000:0.0:1789762875.782902:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.782942:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015075840 00000100:00100000:3.0:1789762875.782950:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015075840 00000100:00100000:3.0:1789762875.782993:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399071546927 time 399071644000 state 0x0 ref 1 00000100:00100000:3.0:1789762875.783005:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399121644000 time 399071644000 state 0x1 ref 1 00000100:00100000:1.0:1789762875.817857:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399121644000 time 399121753251 state 0x0 ref 1 00000100:00100000:1.0:1789762875.817885:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015075840:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.817940:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 82837504, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x27809 00010000:00010000:1.0:1789762875.817947:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e51 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.817960:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762875.817964:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.817969:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.817980:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762875.830938:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968111 is committed 00002000:00010000:1.0:1789762875.830979:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 82837504 -> 83886080 00002000:00010000:1.0:1789762875.830986:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 161801 -> 163849 00010000:00010000:1.0:1789762875.830999:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e51 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.831015:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e51 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.831032:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.831041:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.831046:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e51 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.831056:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e51 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.831076:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e51 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.831106:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e51 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.831191:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015075840:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13313us (48288us total) trans 4294968111 rc 0/0 00000100:00100000:1.0:1789762875.831202:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1005 00000100:00100000:1.0:1789762875.831215:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399121644000 time 399121753251 state 0x2 ref 0 00000100:00100000:2.0:1789762875.833758:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.833807:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015075968 00000100:00100000:3.0:1789762875.833816:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015075968 00000100:00100000:3.0:1789762875.833854:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399121644000 time 399121753251 state 0x0 ref 1 00000100:00100000:3.0:1789762875.833868:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399171753251 time 399121753251 state 0x1 ref 1 00000100:00100000:1.0:1789762875.867948:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399171753251 time 399171842872 state 0x0 ref 1 00000100:00100000:1.0:1789762875.867974:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015075968:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.868024:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 83886080, mtime: 0x6aad9d3b, atime: 0x0, ctime: 0x6aad9d3b, blocks: 0x28009 00010000:00010000:1.0:1789762875.868032:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e58 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.868045:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762875.868049:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.868053:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.868064:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762875.880435:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 83886080 -> 84934656 00002000:00010000:1.0:1789762875.880441:0:12582:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb mtime from disk: 1789762875 -> 1789762876 00000001:00080000:2.0:1789762875.880441:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968112 is committed 00002000:00010000:1.0:1789762875.880445:0:12582:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb ctime from disk: 1789762875 -> 1789762876 00002000:00010000:1.0:1789762875.880450:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 163849 -> 165897 00010000:00010000:1.0:1789762875.880464:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e58 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.880479:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e58 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.880490:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.880499:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.880503:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e58 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.880510:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e58 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.880531:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e58 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.880541:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e58 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.880621:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015075968:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12656us (46860us total) trans 4294968112 rc 0/0 00000100:00100000:1.0:1789762875.880629:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1006 00000100:00100000:1.0:1789762875.880639:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399171753251 time 399171842872 state 0x2 ref 0 00000100:00100000:2.0:1789762875.883365:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762875.883407:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015076096 00000100:00100000:1.0:1789762875.883416:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015076096 00000100:00100000:1.0:1789762875.883458:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399171753251 time 399171842872 state 0x0 ref 1 00000100:00100000:1.0:1789762875.883470:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399221842872 time 399171842872 state 0x1 ref 1 00000100:00100000:1.0:1789762875.918019:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399221842872 time 399221915252 state 0x0 ref 1 00000100:00100000:1.0:1789762875.918045:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015076096:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.918121:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 84934656, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x28809 00010000:00010000:1.0:1789762875.918128:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e5f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.918141:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762875.918145:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.918150:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.918161:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762875.930251:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968113 is committed 00002000:00010000:1.0:1789762875.930270:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:1.0:1789762875.930276:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 165897 -> 167945 00010000:00010000:1.0:1789762875.930288:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e5f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.930299:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e5f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.930305:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.930310:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.930314:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e5f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.930324:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e5f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.930339:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e5f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.930343:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e5f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.930412:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015076096:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12375us (47045us total) trans 4294968113 rc 0/0 00000100:00100000:1.0:1789762875.930420:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1007 00000100:00100000:1.0:1789762875.930430:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399221842872 time 399221915252 state 0x2 ref 0 00000100:00100000:2.0:1789762875.933198:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.933248:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015076224 00000100:00100000:3.0:1789762875.933258:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015076224 00000100:00100000:3.0:1789762875.933306:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399221842872 time 399221915252 state 0x0 ref 1 00000100:00100000:3.0:1789762875.933321:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399271915252 time 399221915252 state 0x1 ref 1 00000100:00100000:1.0:1789762875.968120:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399271915252 time 399272017420 state 0x0 ref 1 00000100:00100000:1.0:1789762875.968144:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015076224:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762875.968200:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 85983232, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x29009 00010000:00010000:1.0:1789762875.968210:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e66 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.968237:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762875.968241:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762875.968246:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.968259:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762875.982003:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968114 is committed 00002000:00010000:1.0:1789762875.982017:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 85983232 -> 87031808 00002000:00010000:1.0:1789762875.982022:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 167945 -> 169993 00010000:00010000:1.0:1789762875.982033:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e66 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.982043:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e66 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.982050:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.982055:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762875.982059:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e66 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.982065:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e66 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.982080:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e66 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762875.982100:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e66 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762875.982170:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015076224:12345-192.168.203.48@tcp:4:dd.0 Request processed in 14034us (48970us total) trans 4294968114 rc 0/0 00000100:00100000:1.0:1789762875.982179:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1008 00000100:00100000:1.0:1789762875.982189:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399271915252 time 399272017420 state 0x2 ref 0 00000100:00100000:1.0:1789762875.984825:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762875.984867:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015076352 00000100:00100000:3.0:1789762875.984876:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015076352 00000100:00100000:3.0:1789762875.984916:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399271915252 time 399272017420 state 0x0 ref 1 00000100:00100000:3.0:1789762875.984927:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399322017420 time 399272017420 state 0x1 ref 1 00000100:00100000:1.0:1789762876.018219:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399322017420 time 399322113971 state 0x0 ref 1 00000100:00100000:1.0:1789762876.018249:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b39500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015076352:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.018304:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 87031808, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x29809 00010000:00010000:1.0:1789762876.018311:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e6d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.018324:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.018328:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.018333:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.018344:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762876.030858:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 87031808 -> 88080384 00002000:00010000:1.0:1789762876.030864:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 169993 -> 172041 00000001:00080000:2.0:1789762876.030873:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968115 is committed 00010000:00010000:1.0:1789762876.030878:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e6d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.030890:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e6d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.030897:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.030903:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.030907:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e6d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.030914:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e6d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.030928:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e6d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.030933:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e6d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.031002:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b39500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015076352:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12759us (46175us total) trans 4294968115 rc 0/0 00000100:00100000:1.0:1789762876.031012:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1009 00000100:00100000:1.0:1789762876.031022:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399322017420 time 399322113971 state 0x2 ref 0 00000100:00100000:0.0:1789762876.033630:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762876.033673:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015076480 00000100:00100000:1.0:1789762876.033682:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015076480 00000100:00100000:1.0:1789762876.033722:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399322017420 time 399322113971 state 0x0 ref 1 00000100:00100000:1.0:1789762876.033733:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399372113971 time 399322113971 state 0x1 ref 1 00000100:00100000:1.0:1789762876.068279:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399372113971 time 399372174625 state 0x0 ref 1 00000100:00100000:1.0:1789762876.068309:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015076480:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.068366:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 88080384, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2a009 00010000:00010000:1.0:1789762876.068374:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e74 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.068389:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.068392:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.068397:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.068408:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762876.082656:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968116 is committed 00002000:00010000:1.0:1789762876.082668:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 88080384 -> 89128960 00002000:00010000:1.0:1789762876.082675:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 172041 -> 174089 00010000:00010000:1.0:1789762876.082686:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e74 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.082697:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e74 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.082705:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.082710:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.082715:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.082725:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e74 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.082744:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.082754:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.082846:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015076480:12345-192.168.203.48@tcp:4:dd.0 Request processed in 14547us (49217us total) trans 4294968116 rc 0/0 00000100:00100000:1.0:1789762876.082856:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1010 00000100:00100000:1.0:1789762876.082866:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399372113971 time 399372174625 state 0x2 ref 0 00000100:00100000:0.0:1789762876.085514:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.085562:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015076608 00000100:00100000:3.0:1789762876.085571:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015076608 00000100:00100000:3.0:1789762876.085612:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399372113971 time 399372174625 state 0x0 ref 1 00000100:00100000:3.0:1789762876.085623:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399422174625 time 399372174625 state 0x1 ref 1 00000100:00100000:1.0:1789762876.118379:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399422174625 time 399422277108 state 0x0 ref 1 00000100:00100000:1.0:1789762876.118404:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015076608:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.118450:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 89128960, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2a809 00010000:00010000:1.0:1789762876.118458:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e7b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.118471:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.118474:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.118479:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.118498:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762876.131516:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968117 is committed 00002000:00010000:1.0:1789762876.131520:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 89128960 -> 90177536 00002000:00010000:1.0:1789762876.131528:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 174089 -> 176137 00010000:00010000:1.0:1789762876.131541:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e7b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.131551:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e7b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.131558:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.131563:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.131566:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e7b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.131577:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e7b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.131592:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e7b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.131597:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e7b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.131663:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015076608:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13268us (46151us total) trans 4294968117 rc 0/0 00000100:00100000:1.0:1789762876.131672:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1011 00000100:00100000:1.0:1789762876.131681:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399422174625 time 399422277108 state 0x2 ref 0 00000100:00100000:0.0:1789762876.134239:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.134310:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015076736 00000100:00100000:3.0:1789762876.134320:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015076736 00000100:00100000:3.0:1789762876.134376:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399422174625 time 399422277108 state 0x0 ref 1 00000100:00100000:3.0:1789762876.134392:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399472277108 time 399422277108 state 0x1 ref 1 00000100:00100000:1.0:1789762876.168532:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399472277108 time 399472425830 state 0x0 ref 1 00000100:00100000:1.0:1789762876.168564:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015076736:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.168620:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 90177536, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2b009 00010000:00010000:1.0:1789762876.168631:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e82 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.168649:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.168655:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.168663:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.168679:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762876.183781:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968118 is committed 00002000:00010000:1.0:1789762876.183813:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 90177536 -> 91226112 00002000:00010000:1.0:1789762876.183821:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 176137 -> 178185 00010000:00010000:1.0:1789762876.183835:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e82 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.183850:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e82 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.183862:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.183871:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.183876:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e82 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.183886:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e82 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.183905:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e82 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.183913:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e82 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.183992:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015076736:12345-192.168.203.48@tcp:4:dd.0 Request processed in 15438us (49753us total) trans 4294968118 rc 0/0 00000100:00100000:1.0:1789762876.184003:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1012 00000100:00100000:1.0:1789762876.184015:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399472277108 time 399472425830 state 0x2 ref 0 00000100:00100000:2.0:1789762876.186335:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762876.186384:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015076864 00000100:00100000:1.0:1789762876.186395:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015076864 00000100:00100000:1.0:1789762876.186462:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399472277108 time 399472425830 state 0x0 ref 1 00000100:00100000:1.0:1789762876.186480:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399522425830 time 399472425830 state 0x1 ref 1 00000100:00100000:1.0:1789762876.218630:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399522425830 time 399522526399 state 0x0 ref 1 00000100:00100000:1.0:1789762876.218657:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015076864:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.218708:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 91226112, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2b809 00010000:00010000:1.0:1789762876.218715:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e89 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.218729:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.218734:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.218741:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.218756:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762876.235307:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968119 is committed 00002000:00010000:1.0:1789762876.235330:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 91226112 -> 92274688 00002000:00010000:1.0:1789762876.235338:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 178185 -> 180233 00010000:00010000:1.0:1789762876.235352:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e89 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.235364:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e89 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.235371:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.235376:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.235379:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e89 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.235402:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e89 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.235419:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e89 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.235424:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e89 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.235511:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015076864:12345-192.168.203.48@tcp:4:dd.0 Request processed in 16861us (49176us total) trans 4294968119 rc 0/0 00000100:00100000:1.0:1789762876.235527:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1013 00000100:00100000:1.0:1789762876.235541:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399522425830 time 399522526399 state 0x2 ref 0 00000100:00100000:0.0:1789762876.238391:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.238445:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015076992 00000100:00100000:3.0:1789762876.238454:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015076992 00000100:00100000:3.0:1789762876.238495:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399522425830 time 399522526399 state 0x0 ref 1 00000100:00100000:3.0:1789762876.238506:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399572526399 time 399522526399 state 0x1 ref 1 00000100:00100000:1.0:1789762876.268758:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399572526399 time 399572648560 state 0x0 ref 1 00000100:00100000:1.0:1789762876.268809:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015076992:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.268861:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 92274688, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2c009 00010000:00010000:1.0:1789762876.268869:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e90 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.268890:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.268893:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.268898:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.268909:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762876.282251:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968120 is committed 00002000:00010000:1.0:1789762876.282269:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:1.0:1789762876.282275:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 180233 -> 182281 00010000:00010000:1.0:1789762876.282291:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e90 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.282303:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e90 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.282316:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.282323:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.282327:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e90 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.282334:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e90 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.282352:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e90 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.282357:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e90 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.282442:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015076992:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13660us (44051us total) trans 4294968120 rc 0/0 00000100:00100000:1.0:1789762876.282452:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1014 00000100:00100000:1.0:1789762876.282465:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399572526399 time 399572648560 state 0x2 ref 0 00000100:00100000:0.0:1789762876.285258:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.285311:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015077120 00000100:00100000:3.0:1789762876.285321:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015077120 00000100:00100000:3.0:1789762876.285371:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399572526399 time 399572648560 state 0x0 ref 1 00000100:00100000:3.0:1789762876.285384:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399622648560 time 399572648560 state 0x1 ref 1 00000100:00100000:1.0:1789762876.318877:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399622648560 time 399622766901 state 0x0 ref 1 00000100:00100000:1.0:1789762876.318909:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015077120:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.318969:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 93323264, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2c809 00010000:00010000:1.0:1789762876.318976:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e97 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.318998:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.319003:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.319009:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.319022:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762876.332312:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968121 is committed 00002000:00010000:1.0:1789762876.332320:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:1.0:1789762876.332326:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 182281 -> 184329 00010000:00010000:1.0:1789762876.332339:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e97 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.332350:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e97 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.332358:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.332363:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.332368:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e97 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.332380:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e97 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.332398:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e97 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.332403:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e97 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.332474:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015077120:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13575us (47217us total) trans 4294968121 rc 0/0 00000100:00100000:1.0:1789762876.332483:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1015 00000100:00100000:1.0:1789762876.332493:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399622648560 time 399622766901 state 0x2 ref 0 00000100:00100000:0.0:1789762876.334998:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762876.335041:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015077248 00000100:00100000:1.0:1789762876.335051:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015077248 00000100:00100000:1.0:1789762876.335114:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399622648560 time 399622766901 state 0x0 ref 1 00000100:00100000:1.0:1789762876.335127:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399672766901 time 399622766901 state 0x1 ref 1 00010000:00010000:0.0:1789762876.343178:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762876.368956:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399672766901 time 399672849840 state 0x0 ref 1 00000100:00100000:1.0:1789762876.368985:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015077248:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.369045:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 94371840, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2d009 00010000:00010000:1.0:1789762876.369055:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e9e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.369071:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.369075:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.369080:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.369119:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762876.382606:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968122 is committed 00002000:00010000:1.0:1789762876.382700:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 94371840 -> 95420416 00002000:00010000:1.0:1789762876.382706:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 184329 -> 186377 00010000:00010000:1.0:1789762876.382722:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e9e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.382734:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e9e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.382741:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.382747:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.382751:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e9e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.382760:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e9e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.382781:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e9e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.382801:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04e9e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.382906:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015077248:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13929us (47906us total) trans 4294968122 rc 0/0 00000100:00100000:1.0:1789762876.382915:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1016 00000100:00100000:1.0:1789762876.382927:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399672766901 time 399672849840 state 0x2 ref 0 00000100:00100000:0.0:1789762876.385712:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.385767:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015077376 00000100:00100000:3.0:1789762876.385781:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015077376 00000100:00100000:3.0:1789762876.385867:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399672766901 time 399672849840 state 0x0 ref 1 00000100:00100000:3.0:1789762876.385890:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399722849840 time 399672849840 state 0x1 ref 1 00000100:00100000:1.0:1789762876.419060:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399722849840 time 399722950593 state 0x0 ref 1 00000100:00100000:1.0:1789762876.419127:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015077376:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.419202:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 95420416, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2d809 00010000:00010000:1.0:1789762876.419212:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04ea5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.419229:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.419234:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.419240:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04ea5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.419254:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04ea5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762876.433130:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968123 is committed 00002000:00010000:3.0:1789762876.433143:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 95420416 -> 96468992 00002000:00010000:3.0:1789762876.433150:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 186377 -> 188427 00010000:00010000:3.0:1789762876.433166:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04ea5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.433181:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04ea5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.433199:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04ea5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.433208:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762876.433214:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04ea5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.433226:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04ea5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.433247:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04ea5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.433256:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ab84054/0x416193e012b04ea5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762876.433338:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015077376:12345-192.168.203.48@tcp:4:dd.0 Request processed in 14258us (47637us total) trans 4294968123 rc 0/0 00000100:00100000:3.0:1789762876.433350:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1017 00000100:00100000:3.0:1789762876.433364:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399722849840 time 399722950593 state 0x2 ref 0 00000100:00100000:0.0:1789762876.435541:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.435583:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015077504 00000100:00100000:3.0:1789762876.435594:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015077504 00000100:00100000:3.0:1789762876.435643:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399722849840 time 399722950593 state 0x0 ref 1 00000100:00100000:3.0:1789762876.435664:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399772950593 time 399722950593 state 0x1 ref 1 00000100:00100000:1.0:1789762876.469184:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399772950593 time 399773074634 state 0x0 ref 1 00000100:00100000:1.0:1789762876.469225:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015077504:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.469295:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 96468992, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2e00b 00010000:00010000:1.0:1789762876.469307:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.469340:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.469344:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.469351:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.469364:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762876.483246:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968124 is committed 00002000:00010000:1.0:1789762876.483252:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 96468992 -> 97517568 00002000:00010000:1.0:1789762876.483258:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 188427 -> 190475 00010000:00010000:1.0:1789762876.483272:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.483288:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.483299:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.483310:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.483316:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.483339:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.483358:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.483364:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.483447:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015077504:12345-192.168.203.48@tcp:4:dd.0 Request processed in 14243us (47905us total) trans 4294968124 rc 0/0 00000100:00100000:1.0:1789762876.483457:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1018 00000100:00100000:1.0:1789762876.483467:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399772950593 time 399773074634 state 0x2 ref 0 00000100:00100000:2.0:1789762876.486526:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762876.486598:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015077632 00000100:00100000:1.0:1789762876.486608:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015077632 00000100:00100000:1.0:1789762876.486666:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399772950593 time 399773074634 state 0x0 ref 1 00000100:00100000:1.0:1789762876.486680:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399823074634 time 399773074634 state 0x1 ref 1 00000100:00100000:1.0:1789762876.519242:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399823074634 time 399823138300 state 0x0 ref 1 00000100:00100000:1.0:1789762876.519268:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015077632:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.519331:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 97517568, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2e80b 00010000:00010000:1.0:1789762876.519339:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eb3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.519355:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.519359:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.519365:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.519377:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762876.532942:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968125 is committed 00002000:00010000:1.0:1789762876.532966:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 97517568 -> 98566144 00002000:00010000:1.0:1789762876.532972:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 190475 -> 192523 00010000:00010000:1.0:1789762876.532985:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eb3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.532997:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eb3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.533004:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.533010:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.533016:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eb3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.533024:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eb3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.533042:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eb3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.533047:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04eb3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.533170:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015077632:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13909us (46643us total) trans 4294968125 rc 0/0 00000100:00100000:1.0:1789762876.533181:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1019 00000100:00100000:1.0:1789762876.533203:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399823074634 time 399823138300 state 0x2 ref 0 00000100:00100000:0.0:1789762876.535992:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.536036:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015077760 00000100:00100000:3.0:1789762876.536047:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015077760 00000100:00100000:3.0:1789762876.536113:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399823074634 time 399823138300 state 0x0 ref 1 00000100:00100000:3.0:1789762876.536127:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399873138300 time 399823138300 state 0x1 ref 1 00000100:00100000:3.0:1789762876.569340:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399873138300 time 399873228002 state 0x0 ref 1 00000100:00100000:3.0:1789762876.569376:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015077760:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:3.0:1789762876.569451:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 98566144, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2f00b 00010000:00010000:3.0:1789762876.569461:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04eba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.569490:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:3.0:1789762876.569494:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762876.569500:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04eba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.569513:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04eba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762876.584046:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968126 is committed 00002000:00010000:1.0:1789762876.584075:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 98566144 -> 99614720 00002000:00010000:1.0:1789762876.584083:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 192523 -> 194571 00010000:00010000:1.0:1789762876.584123:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04eba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.584138:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04eba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.584146:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04eba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.584158:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.584163:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04eba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.584170:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04eba lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.584191:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04eba lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.584197:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b04eba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.584298:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015077760:12345-192.168.203.48@tcp:4:dd.0 Request processed in 14933us (48301us total) trans 4294968126 rc 0/0 00000100:00100000:1.0:1789762876.584309:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1020 00000100:00100000:1.0:1789762876.584321:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399873138300 time 399873228002 state 0x2 ref 0 00000100:00100000:2.0:1789762876.587413:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.587460:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015077888 00000100:00100000:3.0:1789762876.587472:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015077888 00000100:00100000:3.0:1789762876.587525:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399873138300 time 399873228002 state 0x0 ref 1 00000100:00100000:3.0:1789762876.587538:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399923228002 time 399873228002 state 0x1 ref 1 00000100:00100000:1.0:1789762876.619402:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399923228002 time 399923298933 state 0x0 ref 1 00000100:00100000:1.0:1789762876.619426:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015077888:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.619480:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 99614720, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x2f80b 00010000:00010000:1.0:1789762876.619489:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.619505:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.619510:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.619515:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.619528:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762876.632128:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968127 is committed 00002000:00010000:1.0:1789762876.632206:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 99614720 -> 100663296 00002000:00010000:1.0:1789762876.632211:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 194571 -> 196619 00010000:00010000:1.0:1789762876.632224:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.632234:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.632242:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.632247:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.632252:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.632260:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.632282:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.632288:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.632362:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015077888:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12941us (44948us total) trans 4294968127 rc 0/0 00000100:00100000:1.0:1789762876.632371:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1021 00000100:00100000:1.0:1789762876.632381:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399923228002 time 399923298933 state 0x2 ref 0 00000100:00100000:2.0:1789762876.634762:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762876.634809:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015078016 00000100:00100000:1.0:1789762876.634817:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015078016 00000100:00100000:1.0:1789762876.634862:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399923228002 time 399923298933 state 0x0 ref 1 00000100:00100000:1.0:1789762876.634875:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399973298933 time 399923298933 state 0x1 ref 1 00000100:00100000:1.0:1789762876.669476:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399973298933 time 399973369920 state 0x0 ref 1 00000100:00100000:1.0:1789762876.669504:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015078016:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.669564:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 100663296, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x3000b 00010000:00010000:1.0:1789762876.669573:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.669594:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=399s v=(0 0) 00010000:00010000:1.0:1789762876.669598:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.669604:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.669617:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762876.682157:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968128 is committed 00002000:00010000:1.0:1789762876.682245:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 100663296 -> 101711872 00002000:00010000:1.0:1789762876.682252:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 196619 -> 198667 00010000:00010000:1.0:1789762876.682269:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.682282:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.682289:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.682295:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.682299:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.682308:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.682332:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.682338:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ec8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.682437:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015078016:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12940us (47672us total) trans 4294968128 rc 0/0 00000100:00100000:1.0:1789762876.682447:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1022 00000100:00100000:1.0:1789762876.682459:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399973298933 time 399973369920 state 0x2 ref 0 00000100:00100000:2.0:1789762876.685027:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.685077:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015078144 00000100:00100000:3.0:1789762876.685102:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015078144 00000100:00100000:3.0:1789762876.685155:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 399973298933 time 399973369920 state 0x0 ref 1 00000100:00100000:3.0:1789762876.685167:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400023369920 time 399973369920 state 0x1 ref 1 00000100:00100000:1.0:1789762876.719563:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400023369920 time 400023456110 state 0x0 ref 1 00000100:00100000:1.0:1789762876.719595:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015078144:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.719668:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 101711872, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x3080b 00010000:00010000:1.0:1789762876.719677:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ecf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.719694:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=400s v=(0 0) 00010000:00010000:1.0:1789762876.719699:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.719704:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ecf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.719719:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ecf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762876.732192:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968129 is committed 00002000:00010000:1.0:1789762876.732258:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:1.0:1789762876.732263:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 198667 -> 200715 00010000:00010000:1.0:1789762876.732274:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ecf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.732284:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ecf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.732291:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ecf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.732297:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.732301:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ecf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.732309:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ecf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.732325:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ecf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.732330:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ecf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.732401:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015078144:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12820us (47375us total) trans 4294968129 rc 0/0 00000100:00100000:1.0:1789762876.732411:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1023 00000100:00100000:1.0:1789762876.732421:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400023369920 time 400023456110 state 0x2 ref 0 00000100:00100000:2.0:1789762876.734711:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.734752:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015078272 00000100:00100000:3.0:1789762876.734762:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015078272 00000100:00100000:3.0:1789762876.734818:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400023369920 time 400023456110 state 0x0 ref 1 00000100:00100000:3.0:1789762876.734831:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400073456110 time 400023456110 state 0x1 ref 1 00000100:00100000:1.0:1789762876.769696:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400073456110 time 400073578859 state 0x0 ref 1 00000100:00100000:1.0:1789762876.769725:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015078272:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.769802:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 102760448, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x3100b 00010000:00010000:1.0:1789762876.769811:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ed6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.769827:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=400s v=(0 0) 00010000:00010000:1.0:1789762876.769831:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.769837:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ed6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.769851:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ed6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762876.783144:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968130 is committed 00002000:00010000:1.0:1789762876.783158:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:1.0:1789762876.783165:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 200715 -> 202763 00010000:00010000:1.0:1789762876.783180:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ed6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.783193:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ed6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.783204:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ed6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.783213:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.783218:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ed6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.783228:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ed6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.783251:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ed6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.783260:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04ed6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.783343:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015078272:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13628us (48632us total) trans 4294968130 rc 0/0 00000100:00100000:1.0:1789762876.783354:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1024 00000100:00100000:1.0:1789762876.783367:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400073456110 time 400073578859 state 0x2 ref 0 00000100:00100000:0.0:1789762876.787031:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762876.787110:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015078400 00000100:00100000:1.0:1789762876.787121:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015078400 00000100:00100000:1.0:1789762876.787180:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400073456110 time 400073578859 state 0x0 ref 1 00000100:00100000:1.0:1789762876.787202:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400123578859 time 400073578859 state 0x1 ref 1 00000100:00100000:1.0:1789762876.819751:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400123578859 time 400123635211 state 0x0 ref 1 00000100:00100000:1.0:1789762876.819797:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015078400:12345-192.168.203.48@tcp:4:dd.0 00002000:00010000:1.0:1789762876.819865:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 103809024, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x3180b 00010000:00010000:1.0:1789762876.819881:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04edd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.819899:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=400s v=(0 0) 00010000:00010000:1.0:1789762876.819903:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.819909:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04edd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.819922:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04edd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762876.831764:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size from disk: 103809024 -> 104857600 00000001:00080000:0.0:1789762876.831770:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968131 is committed 00002000:00010000:1.0:1789762876.831770:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb blocks from disk: 202763 -> 204811 00010000:00010000:1.0:1789762876.831798:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04edd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.831812:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04edd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.831830:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04edd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.831840:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762876.831847:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04edd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.831856:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04edd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.831876:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04edd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.831882:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b04edd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762876.831944:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015078400:12345-192.168.203.48@tcp:4:dd.0 Request processed in 12174us (44915us total) trans 4294968131 rc 0/0 00000100:00100000:1.0:1789762876.831952:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1025 00000100:00100000:1.0:1789762876.831961:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400123578859 time 400123635211 state 0x2 ref 0 00000100:00100000:2.0:1789762876.833383:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762876.833429:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015078656 00000100:00100000:0.0:1789762876.833438:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015078656 00000100:00080000:2.0:1789762876.833492:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762876.833499:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 100 00000100:00100000:0.0:1789762876.833508:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7b19850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:14831:x1876702015078656:12345-192.168.203.48@tcp:35:dd.0 00000001:00080000:0.0:1789762876.833751:0:11718:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967568 00000100:00100000:0.0:1789762876.833836:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7b19850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14831:x1876702015078656:12345-192.168.203.48@tcp:35:dd.0 Request processed in 330us (455us total) trans 4294967569 rc 0/0 00000100:00100000:0.0:1789762876.833843:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 100 00000020:00080000:0.0:1789762876.833847:0:11718:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1789762876.875880:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762876.875944:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015078912 00000100:00100000:2.0:1789762876.875957:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015078912 00000100:00100000:2.0:1789762876.876016:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 320 00000100:00100000:2.0:1789762876.876026:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14847:x1876702015078912:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762876.876054:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762876.876079:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b04ee4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd712 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.876232:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b04eeb lrc: 3/1,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.876248:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b04eeb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.876263:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b04eeb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762876.876275:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 104857600 00010000:00010000:2.0:1789762876.876399:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b04ee4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd712 expref: 13 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.876414:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b04eeb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd712 expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.876421:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff981ec2179400) 00010000:00010000:2.0:1789762876.876423:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:2.0:1789762876.876441:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4d600, rc 0) 00000100:00100000:2.0:1789762876.876496:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14847:x1876702015078912:12345-192.168.203.48@tcp:101:dd.0 Request processed in 472us (629us total) trans 4294967570 rc 0/0 00000100:00100000:2.0:1789762876.876502:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 320 00000100:00100000:1.0:1789762876.877842:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762876.877888:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015079040 00000100:00100000:2.0:1789762876.877894:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015079040 00000100:00100000:2.0:1789762876.877929:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 185 00000100:00100000:2.0:1789762876.877935:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015079040:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762876.877945:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762876.877958:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04ef9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd719 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762876.877978:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 104857600, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x3200b 00010000:00010000:2.0:1789762876.877993:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=400s v=(0 0) 00010000:00010000:2.0:1789762876.877998:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762876.878005:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04ef9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd719 expref: 6 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.878017:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04ef9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd719 expref: 6 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.878026:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4c400, rc 0) 00000100:00100000:2.0:1789762876.878058:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015079040:12345-192.168.203.48@tcp:101:dd.0 Request processed in 122us (216us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762876.878066:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 185 00000100:00100000:1.0:1789762876.880355:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.880405:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015079168 00000100:00100000:3.0:1789762876.880411:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015079168 00000100:00100000:3.0:1789762876.880458:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 354573509327 time 354473532209 state 0x0 ref 1 00000100:00100000:3.0:1789762876.880475:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 354573532209 time 354473532209 state 0x1 ref 1 00000100:00100000:3.0:1789762876.880505:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 354573532209 time 400184410580 state 0x0 ref 1 00000100:00100000:3.0:1789762876.880518:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015079168:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762876.880542:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f00 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.880555:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762876.880559:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762876.880565:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f00 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.880578:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f00 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.884348:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f00 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.884362:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f00 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.884374:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f00 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.884384:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762876.884391:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f00 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.884403:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f00 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.884421:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f00 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.884431:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f00 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762876.884478:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015079168:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3960us (4121us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762876.884489:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1026 00000100:00100000:3.0:1789762876.884499:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 354573532209 time 400184410580 state 0x2 ref 0 00000100:00100000:1.0:1789762876.886117:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.886148:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015079296 00000100:00100000:3.0:1789762876.886154:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015079296 00000100:00100000:3.0:1789762876.886183:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 354573532209 time 400184410580 state 0x0 ref 1 00000100:00100000:3.0:1789762876.886195:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 2 deadline 400284410580 time 400184410580 state 0x1 ref 1 00000100:00100000:3.0:1789762876.886226:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 400284410580 time 400190132932 state 0x0 ref 1 00000100:00100000:3.0:1789762876.886239:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b38700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015079296:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762876.886257:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f07 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.886269:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762876.886273:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762876.886279:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f07 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.886298:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f07 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.889069:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f07 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.889082:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f07 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.889112:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f07 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.889121:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762876.889126:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f07 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.889138:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f07 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.889152:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f07 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.889162:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f07 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762876.889207:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b38700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015079296:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2969us (3091us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762876.889218:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1027 00000100:00100000:3.0:1789762876.889228:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 400284410580 time 400190132932 state 0x2 ref 0 00000100:00100000:1.0:1789762876.890868:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762876.890925:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015079424 00000100:00100000:2.0:1789762876.890937:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015079424 00000100:00100000:2.0:1789762876.890959:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 400284410580 time 400190132932 state 0x0 ref 1 00000100:00100000:2.0:1789762876.890968:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 1 deadline 400290132932 time 400190132932 state 0x1 ref 1 00000100:00100000:2.0:1789762876.890991:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400290132932 time 400194901218 state 0x0 ref 1 00000100:00100000:2.0:1789762876.891001:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b39c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015079424:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762876.891016:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008d9d91db/0x416193e012b04f0e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.891025:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762876.891027:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762876.891031:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008d9d91db/0x416193e012b04f0e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.891039:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008d9d91db/0x416193e012b04f0e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.893858:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008d9d91db/0x416193e012b04f0e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.893869:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008d9d91db/0x416193e012b04f0e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.893876:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008d9d91db/0x416193e012b04f0e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.893889:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4cc00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762876.893892:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008d9d91db/0x416193e012b04f0e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.893898:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008d9d91db/0x416193e012b04f0e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.893908:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008d9d91db/0x416193e012b04f0e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762876.893914:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008d9d91db/0x416193e012b04f0e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762876.893953:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b39c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015079424:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2953us (3087us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762876.893960:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1028 00000100:00100000:2.0:1789762876.893966:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400290132932 time 400194901218 state 0x2 ref 0 00000100:00100000:0.0:1789762876.895843:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762876.895907:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015079552 00000100:00100000:1.0:1789762876.895913:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015079552 00000100:00100000:1.0:1789762876.895937:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400290132932 time 400194901218 state 0x0 ref 1 00000100:00100000:1.0:1789762876.895946:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400294901218 time 400194901218 state 0x1 ref 1 00000100:00100000:1.0:1789762876.991105:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400294901218 time 400294972799 state 0x0 ref 1 00000100:00100000:1.0:1789762876.991143:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015079552:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762876.991201:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005849ab3b/0x416193e012b04f15 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.991224:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.991229:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762876.991238:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005849ab3b/0x416193e012b04f15 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762876.991266:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005849ab3b/0x416193e012b04f15 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.995072:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005849ab3b/0x416193e012b04f15 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.995091:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005849ab3b/0x416193e012b04f15 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.995129:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005849ab3b/0x416193e012b04f15 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.995138:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a5000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762876.995144:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005849ab3b/0x416193e012b04f15 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.995154:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005849ab3b/0x416193e012b04f15 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.995176:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005849ab3b/0x416193e012b04f15 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762876.995183:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005849ab3b/0x416193e012b04f15 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762876.995261:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015079552:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4130us (99416us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762876.995276:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1029 00000100:00100000:3.0:1789762876.995290:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400294901218 time 400294972799 state 0x2 ref 0 00000100:00100000:2.0:1789762876.997473:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762876.997512:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015079680 00000100:00100000:3.0:1789762876.997521:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015079680 00000100:00100000:3.0:1789762876.997571:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400294901218 time 400294972799 state 0x0 ref 1 00000100:00100000:3.0:1789762876.997588:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400394972799 time 400294972799 state 0x1 ref 1 00000100:00100000:1.0:1789762877.091203:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400394972799 time 400395093843 state 0x0 ref 1 00000100:00100000:1.0:1789762877.091251:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015079680:12345-192.168.203.48@tcp:3:dd.0 00000100:00080000:2.0:1789762877.091283:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762877.091301:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000913ff8c1/0x416193e012b04f1c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.091323:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762877.091327:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762877.091334:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000913ff8c1/0x416193e012b04f1c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.091347:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000913ff8c1/0x416193e012b04f1c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.094249:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000913ff8c1/0x416193e012b04f1c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.094260:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000913ff8c1/0x416193e012b04f1c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.094271:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000913ff8c1/0x416193e012b04f1c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.094280:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a4600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762877.094286:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000913ff8c1/0x416193e012b04f1c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.094296:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000913ff8c1/0x416193e012b04f1c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.094318:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000913ff8c1/0x416193e012b04f1c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.094331:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000913ff8c1/0x416193e012b04f1c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762877.094388:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015079680:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3169us (96915us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762877.094395:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1030 00000100:00100000:1.0:1789762877.094403:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400394972799 time 400395093843 state 0x2 ref 0 00000100:00100000:0.0:1789762877.097254:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762877.097295:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015079808 00000100:00100000:3.0:1789762877.097307:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015079808 00000100:00100000:3.0:1789762877.097363:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400394972799 time 400395093843 state 0x0 ref 1 00000100:00100000:3.0:1789762877.097378:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400495093843 time 400395093843 state 0x1 ref 1 00000100:00100000:2.0:1789762877.191284:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400495093843 time 400495174904 state 0x0 ref 1 00000100:00100000:2.0:1789762877.191314:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015079808:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762877.191369:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b04f23 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.191403:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762877.191407:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762877.191413:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b04f23 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.191426:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b04f23 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.194606:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b04f23 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.194621:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b04f23 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.194629:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b04f23 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.194635:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762877.194640:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b04f23 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.194647:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b04f23 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.194665:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b04f23 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.194682:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b04f23 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762877.194750:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015079808:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3447us (97502us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762877.194759:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1031 00000100:00100000:2.0:1789762877.194767:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400495093843 time 400495174904 state 0x2 ref 0 00000100:00100000:2.0:1789762877.196694:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762877.196739:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015079936 00000100:00100000:1.0:1789762877.196749:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015079936 00000100:00100000:1.0:1789762877.196811:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400495093843 time 400495174904 state 0x0 ref 1 00000100:00100000:1.0:1789762877.196829:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400595174904 time 400495174904 state 0x1 ref 1 00000100:00100000:3.0:1789762877.291398:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400595174904 time 400595294029 state 0x0 ref 1 00000100:00100000:3.0:1789762877.291424:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015079936:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762877.291463:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f2a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.291478:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762877.291481:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762877.291486:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f2a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.291501:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f2a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.294531:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f2a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.294542:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f2a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.294549:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f2a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.294554:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762877.294557:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f2a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.294564:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f2a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.294576:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f2a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.294581:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f2a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762877.294632:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015079936:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3218us (97938us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762877.294639:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1032 00000100:00100000:3.0:1789762877.294647:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400595174904 time 400595294029 state 0x2 ref 0 00000100:00100000:1.0:1789762877.298421:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762877.298465:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015080064 00000100:00100000:3.0:1789762877.298474:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015080064 00000100:00100000:3.0:1789762877.298508:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400595174904 time 400595294029 state 0x0 ref 1 00000100:00100000:3.0:1789762877.298519:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400695294029 time 400595294029 state 0x1 ref 1 00010000:00010000:2.0:1789762877.367258:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762877.391474:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400695294029 time 400695370089 state 0x0 ref 1 00000100:00100000:1.0:1789762877.391508:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b39f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015080064:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762877.391544:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04f31 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.391558:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762877.391562:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762877.391567:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04f31 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.391579:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04f31 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.394696:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04f31 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.394707:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04f31 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.394714:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04f31 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.394720:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762877.394724:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04f31 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.394731:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04f31 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.394745:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04f31 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.394751:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b04f31 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762877.394836:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b39f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015080064:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3344us (96414us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762877.394849:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1033 00000100:00100000:3.0:1789762877.394860:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400695294029 time 400695370089 state 0x2 ref 0 00000100:00100000:0.0:1789762877.397120:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762877.397159:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015080192 00000100:00100000:3.0:1789762877.397167:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015080192 00000100:00100000:3.0:1789762877.397206:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400695294029 time 400695370089 state 0x0 ref 1 00000100:00100000:3.0:1789762877.397221:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400795370089 time 400695370089 state 0x1 ref 1 00000100:00100000:2.0:1789762877.491584:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400795370089 time 400795474611 state 0x0 ref 1 00000100:00100000:2.0:1789762877.491611:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015080192:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762877.491658:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b04f38 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.491676:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762877.491680:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762877.491685:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b04f38 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.491707:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b04f38 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.495320:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b04f38 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.495330:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b04f38 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.495338:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b04f38 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.495344:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762877.495348:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b04f38 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.495355:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b04f38 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.495371:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b04f38 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.495376:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b04f38 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762877.495436:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015080192:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3836us (98321us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762877.495445:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1034 00000100:00100000:2.0:1789762877.495461:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400795370089 time 400795474611 state 0x2 ref 0 00000100:00100000:1.0:1789762877.498386:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762877.498463:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015080320 00000100:00100000:2.0:1789762877.498472:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015080320 00000100:00100000:2.0:1789762877.498516:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400795370089 time 400795474611 state 0x0 ref 1 00000100:00100000:2.0:1789762877.498529:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400895474611 time 400795474611 state 0x1 ref 1 00000001:00080000:0.0:1789762877.568644:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967569 is committed 00000001:00080000:0.0:1789762877.568660:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967570 is committed 00000100:00100000:3.0:1789762877.591699:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400895474611 time 400895596318 state 0x0 ref 1 00000100:00100000:3.0:1789762877.591722:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b39880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015080320:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762877.591758:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f3f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.591772:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762877.591797:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762877.591803:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f3f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.591819:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f3f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.594979:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f3f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.594988:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f3f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.594994:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f3f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.594999:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762877.595003:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f3f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.595009:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f3f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.595024:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f3f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.595030:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f3f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762877.595133:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b39880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015080320:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3417us (96752us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762877.595141:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1035 00000100:00100000:3.0:1789762877.595151:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400895474611 time 400895596318 state 0x2 ref 0 00000100:00100000:1.0:1789762877.597016:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762877.597050:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015080448 00000100:00100000:3.0:1789762877.597057:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015080448 00000100:00100000:3.0:1789762877.597112:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400895474611 time 400895596318 state 0x0 ref 1 00000100:00100000:3.0:1789762877.597127:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400995596318 time 400895596318 state 0x1 ref 1 00000100:00100000:3.0:1789762877.691744:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400995596318 time 400995639843 state 0x0 ref 1 00000100:00100000:3.0:1789762877.691769:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b38380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015080448:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762877.691821:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f46 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.691835:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762877.691838:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762877.691843:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f46 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.691855:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f46 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.695132:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f46 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.695142:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f46 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.695151:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f46 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.695157:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762877.695161:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f46 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.695168:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f46 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.695181:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f46 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.695186:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f46 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762877.695242:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b38380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015080448:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3481us (98226us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762877.695253:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1036 00000100:00100000:3.0:1789762877.695260:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400995596318 time 400995639843 state 0x2 ref 0 00000100:00100000:1.0:1789762877.697322:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762877.697349:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015080576 00000100:00100000:3.0:1789762877.697358:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015080576 00000100:00100000:3.0:1789762877.697392:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 400995596318 time 400995639843 state 0x0 ref 1 00000100:00100000:3.0:1789762877.697402:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401095639843 time 400995639843 state 0x1 ref 1 00000100:00100000:2.0:1789762877.791887:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401095639843 time 401095777243 state 0x0 ref 1 00000100:00100000:2.0:1789762877.791921:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81b3aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015080576:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762877.791974:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d3a72c98/0x416193e012b04f4d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.791992:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762877.791996:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762877.792001:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d3a72c98/0x416193e012b04f4d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762877.792018:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d3a72c98/0x416193e012b04f4d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.795295:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d3a72c98/0x416193e012b04f4d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.795311:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d3a72c98/0x416193e012b04f4d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.795318:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d3a72c98/0x416193e012b04f4d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.795323:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d864c9e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762877.795327:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3a72c98/0x416193e012b04f4d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.795333:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d3a72c98/0x416193e012b04f4d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.795347:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d3a72c98/0x416193e012b04f4d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762877.795356:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d3a72c98/0x416193e012b04f4d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762877.795446:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81b3aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015080576:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3535us (98119us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762877.795460:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1037 00000100:00100000:1.0:1789762877.795472:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401095639843 time 401095777243 state 0x2 ref 0 00000100:00100000:1.0:1789762877.797423:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762877.797478:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015080704 00000100:00100000:2.0:1789762877.797488:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015080704 00000100:00100000:2.0:1789762877.797535:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401095639843 time 401095777243 state 0x0 ref 1 00000100:00100000:2.0:1789762877.797559:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401195777243 time 401095777243 state 0x1 ref 1 00000100:00100000:3.0:1789762877.891986:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401195777243 time 401195882076 state 0x0 ref 1 00000100:00100000:3.0:1789762877.892021:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015080704:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762877.892057:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f54 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.892072:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762877.892075:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762877.892080:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f54 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.892121:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f54 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.895382:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f54 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.895403:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f54 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.895417:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f54 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.895434:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762877.895439:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f54 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.895448:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f54 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.895467:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f54 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.895472:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f54 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762877.895547:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015080704:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3542us (98122us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762877.895556:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1038 00000100:00100000:3.0:1789762877.895567:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401195777243 time 401195882076 state 0x2 ref 0 00000100:00100000:0.0:1789762877.897689:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762877.897724:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015080832 00000100:00100000:3.0:1789762877.897735:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015080832 00000100:00100000:3.0:1789762877.897794:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401195777243 time 401195882076 state 0x0 ref 1 00000100:00100000:3.0:1789762877.897810:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401295882076 time 401195882076 state 0x1 ref 1 00000100:00100000:3.0:1789762877.992074:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401295882076 time 401295963515 state 0x0 ref 1 00000100:00100000:3.0:1789762877.992130:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015080832:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762877.992178:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f5b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.992196:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762877.992201:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762877.992207:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f5b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.992220:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f5b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.995767:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f5b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.995789:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f5b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.995798:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f5b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.995808:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762877.995813:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f5b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.995820:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f5b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.995835:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f5b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762877.995840:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f5b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762877.995896:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015080832:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3784us (98209us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762877.995904:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1039 00000100:00100000:3.0:1789762877.995913:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401295882076 time 401295963515 state 0x2 ref 0 00000100:00100000:2.0:1789762877.997745:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762877.997810:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015080960 00000100:00100000:3.0:1789762877.997830:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015080960 00000100:00100000:3.0:1789762877.997880:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401295882076 time 401295963515 state 0x0 ref 1 00000100:00100000:3.0:1789762877.997893:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401395963515 time 401295963515 state 0x1 ref 1 00000100:00100000:1.0:1789762878.092194:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401395963515 time 401396077553 state 0x0 ref 1 00000100:00100000:1.0:1789762878.092233:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015080960:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762878.092286:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fbb6dbeb/0x416193e012b04f62 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.092304:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762878.092308:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762878.092314:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fbb6dbeb/0x416193e012b04f62 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.092327:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fbb6dbeb/0x416193e012b04f62 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.095964:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fbb6dbeb/0x416193e012b04f62 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.095978:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fbb6dbeb/0x416193e012b04f62 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.095987:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fbb6dbeb/0x416193e012b04f62 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.095993:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa43c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762878.095998:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fbb6dbeb/0x416193e012b04f62 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.096005:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fbb6dbeb/0x416193e012b04f62 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.096024:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fbb6dbeb/0x416193e012b04f62 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.096031:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fbb6dbeb/0x416193e012b04f62 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762878.096120:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015080960:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3875us (98352us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762878.096145:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1040 00000100:00100000:2.0:1789762878.096154:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401395963515 time 401396077553 state 0x2 ref 0 00000100:00100000:3.0:1789762878.098251:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762878.098288:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015081088 00000100:00100000:2.0:1789762878.098298:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015081088 00000100:00100000:2.0:1789762878.098344:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401395963515 time 401396077553 state 0x0 ref 1 00000100:00100000:2.0:1789762878.098357:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401496077553 time 401396077553 state 0x1 ref 1 00000100:00100000:3.0:1789762878.192313:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401496077553 time 401496198965 state 0x0 ref 1 00000100:00100000:3.0:1789762878.192356:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015081088:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762878.192410:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f69 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.192440:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762878.192446:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762878.192454:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f69 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.192471:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f69 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.196136:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f69 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.196150:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f69 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.196158:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f69 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.196164:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762878.196168:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f69 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.196176:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f69 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.196193:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f69 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.196198:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b04f69 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762878.196260:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015081088:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3923us (98009us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762878.196267:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1041 00000100:00100000:1.0:1789762878.196281:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401496077553 time 401496198965 state 0x2 ref 0 00000100:00100000:0.0:1789762878.198385:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762878.198434:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015081216 00000100:00100000:3.0:1789762878.198445:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015081216 00000100:00100000:3.0:1789762878.198494:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401496077553 time 401496198965 state 0x0 ref 1 00000100:00100000:3.0:1789762878.198511:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401596198965 time 401496198965 state 0x1 ref 1 00000100:00100000:3.0:1789762878.292377:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401596198965 time 401596268230 state 0x0 ref 1 00000100:00100000:3.0:1789762878.292410:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015081216:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762878.292463:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f70 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.292482:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762878.292485:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762878.292490:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f70 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.292500:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f70 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.295635:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f70 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.295646:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f70 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.295653:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f70 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.295659:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762878.295663:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f70 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.295670:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f70 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.295686:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f70 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.295695:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b04f70 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762878.295746:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015081216:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3349us (97363us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762878.295752:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1042 00000100:00100000:3.0:1789762878.295760:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401596198965 time 401596268230 state 0x2 ref 0 00000100:00100000:2.0:1789762878.297643:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762878.297678:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015081344 00000100:00100000:3.0:1789762878.297687:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015081344 00000100:00100000:3.0:1789762878.297723:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401596198965 time 401596268230 state 0x0 ref 1 00000100:00100000:3.0:1789762878.297733:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401696268230 time 401596268230 state 0x1 ref 1 00010000:00010000:3.0:1789762878.391271:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762878.392440:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401696268230 time 401696335868 state 0x0 ref 1 00000100:00100000:2.0:1789762878.392465:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015081344:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762878.392502:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000760e1ac1/0x416193e012b04f77 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.392516:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762878.392520:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762878.392525:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000760e1ac1/0x416193e012b04f77 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.392537:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000760e1ac1/0x416193e012b04f77 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.395682:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000760e1ac1/0x416193e012b04f77 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.395703:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000760e1ac1/0x416193e012b04f77 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.395710:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000760e1ac1/0x416193e012b04f77 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.395716:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa41000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762878.395719:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000760e1ac1/0x416193e012b04f77 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.395725:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000760e1ac1/0x416193e012b04f77 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.395738:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000760e1ac1/0x416193e012b04f77 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.395743:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000760e1ac1/0x416193e012b04f77 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762878.395808:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015081344:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3351us (98165us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762878.395815:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1043 00000100:00100000:2.0:1789762878.395822:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401696268230 time 401696335868 state 0x2 ref 0 00000100:00100000:2.0:1789762878.397788:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762878.397847:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015081472 00000100:00100000:1.0:1789762878.397861:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015081472 00000100:00100000:1.0:1789762878.397922:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401696268230 time 401696335868 state 0x0 ref 1 00000100:00100000:1.0:1789762878.397935:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401796335868 time 401696335868 state 0x1 ref 1 00000100:00100000:1.0:1789762878.492538:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401796335868 time 401796427370 state 0x0 ref 1 00000100:00100000:1.0:1789762878.492575:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015081472:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762878.492635:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f7e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.492651:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762878.492656:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762878.492662:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f7e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.492675:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f7e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.495979:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f7e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.495990:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f7e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.495997:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f7e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.496004:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762878.496009:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f7e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.496017:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f7e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.496038:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f7e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.496047:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f7e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762878.496161:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015081472:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3602us (98386us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762878.496172:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1044 00000100:00100000:1.0:1789762878.496181:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401796335868 time 401796427370 state 0x2 ref 0 00000100:00100000:2.0:1789762878.498190:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762878.498231:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015081600 00000100:00100000:3.0:1789762878.498241:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015081600 00000100:00100000:3.0:1789762878.498290:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401796335868 time 401796427370 state 0x0 ref 1 00000100:00100000:3.0:1789762878.498303:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401896427370 time 401796427370 state 0x1 ref 1 00000100:00100000:1.0:1789762878.592629:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401896427370 time 401896519938 state 0x0 ref 1 00000100:00100000:1.0:1789762878.592664:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015081600:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762878.592713:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f85 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.592734:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762878.592739:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762878.592744:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f85 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.592758:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f85 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.595826:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f85 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.595843:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f85 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.595851:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f85 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.595857:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762878.595861:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f85 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.595867:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f85 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.595883:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f85 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.595892:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b04f85 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762878.595969:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015081600:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3321us (97784us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762878.595979:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1045 00000100:00100000:3.0:1789762878.595989:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401896427370 time 401896519938 state 0x2 ref 0 00000100:00100000:2.0:1789762878.597944:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762878.597980:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015081728 00000100:00100000:3.0:1789762878.597989:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015081728 00000100:00100000:3.0:1789762878.598026:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401896427370 time 401896519938 state 0x0 ref 1 00000100:00100000:3.0:1789762878.598039:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401996519938 time 401896519938 state 0x1 ref 1 00000100:00100000:2.0:1789762878.692719:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401996519938 time 401996612576 state 0x0 ref 1 00000100:00100000:2.0:1789762878.692751:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015081728:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762878.692809:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006fd85831/0x416193e012b04f8c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.692828:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762878.692832:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762878.692839:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006fd85831/0x416193e012b04f8c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762878.692854:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006fd85831/0x416193e012b04f8c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.695935:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006fd85831/0x416193e012b04f8c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.695947:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006fd85831/0x416193e012b04f8c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.695955:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006fd85831/0x416193e012b04f8c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.695961:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa40600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762878.695965:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006fd85831/0x416193e012b04f8c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.695971:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006fd85831/0x416193e012b04f8c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.695986:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006fd85831/0x416193e012b04f8c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.695997:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006fd85831/0x416193e012b04f8c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762878.696047:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015081728:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3309us (98104us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762878.696054:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1046 00000100:00100000:1.0:1789762878.696062:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401996519938 time 401996612576 state 0x2 ref 0 00000100:00100000:0.0:1789762878.697987:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762878.698034:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015081856 00000100:00100000:1.0:1789762878.698043:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015081856 00000100:00100000:1.0:1789762878.698082:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 401996519938 time 401996612576 state 0x0 ref 1 00000100:00100000:1.0:1789762878.698114:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402096612576 time 401996612576 state 0x1 ref 1 00000100:00100000:1.0:1789762878.792766:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402096612576 time 402096663160 state 0x0 ref 1 00000100:00100000:1.0:1789762878.792806:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015081856:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762878.792858:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04f93 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.792877:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762878.792881:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762878.792886:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04f93 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.792904:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04f93 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.796329:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04f93 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.796339:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04f93 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.796345:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04f93 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.796359:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762878.796363:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04f93 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.796373:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04f93 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.796385:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04f93 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.796390:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04f93 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762878.796457:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdff100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015081856:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3661us (98483us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762878.796468:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1047 00000100:00100000:1.0:1789762878.796478:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402096612576 time 402096663160 state 0x2 ref 0 00000100:00100000:2.0:1789762878.798558:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762878.798602:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015081984 00000100:00100000:3.0:1789762878.798611:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015081984 00000100:00100000:3.0:1789762878.798657:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402096612576 time 402096663160 state 0x0 ref 1 00000100:00100000:3.0:1789762878.798668:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402196663160 time 402096663160 state 0x1 ref 1 00000100:00100000:3.0:1789762878.892838:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402196663160 time 402196733498 state 0x0 ref 1 00000100:00100000:3.0:1789762878.892865:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015081984:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762878.892908:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04f9a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.892926:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762878.892931:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762878.892937:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04f9a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.892949:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04f9a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.896180:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04f9a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.896193:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04f9a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.896199:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04f9a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.896205:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762878.896209:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04f9a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.896215:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04f9a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.896230:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04f9a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762878.896236:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04f9a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762878.896292:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015081984:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3437us (97735us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762878.896301:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1048 00000100:00100000:3.0:1789762878.896309:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402196663160 time 402196733498 state 0x2 ref 0 00000100:00100000:2.0:1789762878.898498:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762878.898530:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015082112 00000100:00100000:3.0:1789762878.898538:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015082112 00000100:00100000:3.0:1789762878.898571:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402196663160 time 402196733498 state 0x0 ref 1 00000100:00100000:3.0:1789762878.898581:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402296733498 time 402196733498 state 0x1 ref 1 00000100:00100000:1.0:1789762878.992910:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402296733498 time 402296805896 state 0x0 ref 1 00000100:00100000:1.0:1789762878.992935:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015082112:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762878.992983:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.992997:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762878.993001:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762878.993005:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.993017:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.996048:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.996058:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.996069:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.996074:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762878.996077:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.996083:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.996112:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762878.996117:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762878.996168:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015082112:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3243us (97670us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762878.996175:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1049 00000100:00100000:1.0:1789762878.996182:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402296733498 time 402296805896 state 0x2 ref 0 00000100:00100000:0.0:1789762878.998252:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762878.998288:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015082240 00000100:00100000:1.0:1789762878.998297:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015082240 00000100:00100000:1.0:1789762878.998331:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402296733498 time 402296805896 state 0x0 ref 1 00000100:00100000:1.0:1789762878.998345:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402396805896 time 402296805896 state 0x1 ref 1 00000100:00100000:1.0:1789762879.092980:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402396805896 time 402396874081 state 0x0 ref 1 00000100:00100000:1.0:1789762879.093030:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015082240:12345-192.168.203.48@tcp:3:dd.0 00000100:00080000:2.0:1789762879.093052:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762879.093070:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.093112:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762879.093116:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762879.093121:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.093133:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.096178:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.096189:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.096196:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.096202:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762879.096206:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.096212:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.096228:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.096233:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b04fa8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762879.096287:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015082240:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3289us (98037us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762879.096296:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1050 00000100:00100000:3.0:1789762879.096303:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402396805896 time 402396874081 state 0x2 ref 0 00000100:00100000:2.0:1789762879.098167:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762879.098200:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015082368 00000100:00100000:3.0:1789762879.098207:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015082368 00000100:00100000:3.0:1789762879.098241:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402396805896 time 402396874081 state 0x0 ref 1 00000100:00100000:3.0:1789762879.098252:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402496874081 time 402396874081 state 0x1 ref 1 00000100:00100000:3.0:1789762879.193041:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402496874081 time 402496935881 state 0x0 ref 1 00000100:00100000:3.0:1789762879.193066:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015082368:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762879.193123:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04faf lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.193138:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762879.193141:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762879.193145:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04faf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.193156:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04faf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.196215:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04faf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.196226:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04faf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.196233:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04faf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.196238:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762879.196242:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04faf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.196248:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04faf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.196261:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04faf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.196266:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04faf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762879.196316:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015082368:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3260us (98150us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762879.196323:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1051 00000100:00100000:3.0:1789762879.196332:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402496874081 time 402496935881 state 0x2 ref 0 00000100:00100000:1.0:1789762879.198242:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762879.198282:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015082496 00000100:00100000:3.0:1789762879.198289:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015082496 00000100:00100000:3.0:1789762879.198322:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402496874081 time 402496935881 state 0x0 ref 1 00000100:00100000:3.0:1789762879.198332:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402596935881 time 402496935881 state 0x1 ref 1 00000100:00100000:1.0:1789762879.293568:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402596935881 time 402597458243 state 0x0 ref 1 00000100:00100000:1.0:1789762879.293600:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015082496:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762879.293642:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000062b0cb1/0x416193e012b04fb6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.293659:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762879.293665:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762879.293672:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000062b0cb1/0x416193e012b04fb6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.293686:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000062b0cb1/0x416193e012b04fb6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.296912:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000062b0cb1/0x416193e012b04fb6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.296925:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000062b0cb1/0x416193e012b04fb6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.296932:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000062b0cb1/0x416193e012b04fb6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.296946:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87a84000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762879.296949:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000062b0cb1/0x416193e012b04fb6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.296956:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000062b0cb1/0x416193e012b04fb6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.296969:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000062b0cb1/0x416193e012b04fb6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.296975:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000062b0cb1/0x416193e012b04fb6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762879.297024:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015082496:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3437us (98782us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762879.297032:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1052 00000100:00100000:2.0:1789762879.297041:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402596935881 time 402597458243 state 0x2 ref 0 00000100:00100000:2.0:1789762879.299149:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762879.299196:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015082624 00000100:00100000:1.0:1789762879.299206:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015082624 00000100:00100000:1.0:1789762879.299257:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402596935881 time 402597458243 state 0x0 ref 1 00000100:00100000:1.0:1789762879.299273:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402697458243 time 402597458243 state 0x1 ref 1 00000100:00100000:3.0:1789762879.393639:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402697458243 time 402697534038 state 0x0 ref 1 00000100:00100000:3.0:1789762879.393659:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015082624:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762879.393695:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04fbd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.393710:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762879.393713:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762879.393718:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04fbd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.393728:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04fbd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.396418:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04fbd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.396429:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04fbd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.396436:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04fbd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.396442:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762879.396445:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04fbd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.396451:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04fbd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.396465:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04fbd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.396471:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b04fbd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762879.396527:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015082624:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2873us (97379us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762879.396534:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1053 00000100:00100000:1.0:1789762879.396541:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402697458243 time 402697534038 state 0x2 ref 0 00000100:00100000:2.0:1789762879.398506:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762879.398549:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015082752 00000100:00100000:3.0:1789762879.398557:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015082752 00000100:00100000:3.0:1789762879.398592:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402697458243 time 402697534038 state 0x0 ref 1 00000100:00100000:3.0:1789762879.398606:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402797534038 time 402697534038 state 0x1 ref 1 00010000:00010000:0.0:1789762879.415196:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762879.493688:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402797534038 time 402797583087 state 0x0 ref 1 00000100:00100000:3.0:1789762879.493712:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015082752:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762879.493748:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fc4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.493775:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762879.493780:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762879.493788:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fc4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.493812:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fc4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.497363:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fc4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.497379:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fc4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.497391:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fc4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.497407:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762879.497413:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fc4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.497422:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fc4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.497448:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fc4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.497454:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fc4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762879.497519:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015082752:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3815us (99013us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762879.497527:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1054 00000100:00100000:3.0:1789762879.497535:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402797534038 time 402797583087 state 0x2 ref 0 00000100:00100000:0.0:1789762879.499516:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762879.499550:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015082880 00000100:00100000:3.0:1789762879.499558:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015082880 00000100:00100000:3.0:1789762879.499596:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402797534038 time 402797583087 state 0x0 ref 1 00000100:00100000:3.0:1789762879.499612:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402897583087 time 402797583087 state 0x1 ref 1 00000100:00100000:2.0:1789762879.586341:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762879.586391:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762879.586401:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015083136 00000100:00100000:0.0:1789762879.586410:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015083136 00000100:00100000:3.0:1789762879.586440:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015083008 00000100:00100000:2.0:1789762879.586445:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762879.586449:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015083008 00000100:00080000:1.0:1789762879.586479:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762879.586481:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015083264 00000100:00100000:0.0:1789762879.586486:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015083264 00000100:00100000:2.0:1789762879.586498:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 313 00000100:00100000:3.0:1789762879.586498:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 330 00000100:00100000:3.0:1789762879.586514:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850ddc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015083008:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762879.586522:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702015083136:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762879.586527:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 314 00000100:00100000:0.0:1789762879.586541:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015083264:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:1.0:1789762879.586541:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762879.586594:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850ddc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015083008:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 84us (204us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762879.586602:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015083264:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 65us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762879.586606:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702015083136:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 90us (270us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762879.586606:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 330 00000100:00100000:0.0:1789762879.586617:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 314 00000100:00100000:2.0:1789762879.586627:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 313 00000100:00100000:2.0:1789762879.593782:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402897583087 time 402897668690 state 0x0 ref 1 00000100:00100000:2.0:1789762879.593800:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015082880:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762879.593832:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04fcb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.593850:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762879.593854:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762879.593861:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04fcb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.593885:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04fcb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.597228:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04fcb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.597241:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04fcb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.597251:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04fcb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.597272:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86184200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762879.597277:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04fcb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.597287:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04fcb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.597305:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04fcb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.597313:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7e7a810/0x416193e012b04fcb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762879.597363:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015082880:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3566us (97846us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762879.597374:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1055 00000100:00100000:2.0:1789762879.597384:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402897583087 time 402897668690 state 0x2 ref 0 00000100:00100000:2.0:1789762879.599271:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762879.599352:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015083392 00000100:00100000:1.0:1789762879.599358:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015083392 00000100:00100000:1.0:1789762879.599391:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402897583087 time 402897668690 state 0x0 ref 1 00000100:00100000:1.0:1789762879.599402:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402997668690 time 402897668690 state 0x1 ref 1 00000100:00100000:3.0:1789762879.607235:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb17b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022181120:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762879.607278:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762879.607323:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022181120 00000100:00100000:2.0:1789762879.607330:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022181120 00000100:00080000:1.0:1789762879.607366:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762879.607389:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:2.0:1789762879.607400:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022181120:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762879.607426:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762879.607475:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022181120:12345-0@lo:13:osp-pre-0-0.0 Request processed in 76us (193us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762879.607493:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00100000:3.0:1789762879.607505:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb17b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022181120:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762879.693839:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402997668690 time 402997735632 state 0x0 ref 1 00000100:00100000:1.0:1789762879.693864:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015083392:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762879.693901:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04fd2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.693916:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762879.693919:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762879.693924:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04fd2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.693934:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04fd2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.696633:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04fd2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.696647:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04fd2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.696658:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04fd2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.696667:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762879.696673:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04fd2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.696684:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04fd2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.696702:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04fd2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.696714:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b04fd2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762879.696761:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015083392:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2907us (97491us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762879.696783:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1056 00000100:00100000:1.0:1789762879.696791:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402997668690 time 402997735632 state 0x2 ref 0 00000100:00100000:2.0:1789762879.698793:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762879.698836:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015083520 00000100:00100000:3.0:1789762879.698845:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015083520 00000100:00100000:3.0:1789762879.698881:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 402997668690 time 402997735632 state 0x0 ref 1 00000100:00100000:3.0:1789762879.698892:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403097735632 time 402997735632 state 0x1 ref 1 00000100:00100000:3.0:1789762879.793904:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403097735632 time 403097800627 state 0x0 ref 1 00000100:00100000:3.0:1789762879.793929:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb27480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015083520:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762879.793966:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fd9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.793980:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762879.793984:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762879.793988:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fd9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.794005:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fd9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.797348:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fd9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.797360:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fd9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.797366:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fd9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.797371:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762879.797374:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fd9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.797381:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fd9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.797393:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fd9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.797398:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fd9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762879.797449:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb27480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015083520:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3530us (98657us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762879.797456:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1057 00000100:00100000:3.0:1789762879.797464:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403097735632 time 403097800627 state 0x2 ref 0 00000100:00100000:2.0:1789762879.799407:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762879.799433:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015083648 00000100:00100000:3.0:1789762879.799441:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015083648 00000100:00100000:3.0:1789762879.799474:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403097735632 time 403097800627 state 0x0 ref 1 00000100:00100000:3.0:1789762879.799485:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403197800627 time 403097800627 state 0x1 ref 1 00000100:00100000:2.0:1789762879.893985:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403197800627 time 403197880848 state 0x0 ref 1 00000100:00100000:2.0:1789762879.894009:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015083648:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762879.894045:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04fe0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.894058:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762879.894062:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762879.894066:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04fe0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.894077:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04fe0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.897113:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04fe0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.897126:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04fe0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.897137:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04fe0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.897163:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87cbe600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762879.897168:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04fe0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.897176:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04fe0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.897189:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04fe0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762879.897194:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082c288f9/0x416193e012b04fe0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762879.897243:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015083648:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3242us (97837us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762879.897254:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1058 00000100:00100000:2.0:1789762879.897266:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403197800627 time 403197880848 state 0x2 ref 0 00000100:00100000:2.0:1789762879.899129:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762879.899189:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015083776 00000100:00100000:1.0:1789762879.899198:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015083776 00000100:00100000:1.0:1789762879.899233:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403197800627 time 403197880848 state 0x0 ref 1 00000100:00100000:1.0:1789762879.899243:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403297880848 time 403197880848 state 0x1 ref 1 00000100:00100000:1.0:1789762879.994057:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403297880848 time 403297953445 state 0x0 ref 1 00000100:00100000:1.0:1789762879.994083:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015083776:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762879.994147:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b04fe7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.994162:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762879.994165:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762879.994170:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b04fe7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762879.994185:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b04fe7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.997184:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b04fe7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.997195:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b04fe7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.997201:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b04fe7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.997206:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec05c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762879.997209:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b04fe7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.997215:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b04fe7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.997229:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b04fe7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762879.997234:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d165cc67/0x416193e012b04fe7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762879.997279:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015083776:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3206us (98151us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762879.997286:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1059 00000100:00100000:3.0:1789762879.997293:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403297880848 time 403297953445 state 0x2 ref 0 00000100:00100000:1.0:1789762879.999242:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762879.999277:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015083904 00000100:00100000:3.0:1789762879.999285:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015083904 00000100:00100000:3.0:1789762879.999321:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403297880848 time 403297953445 state 0x0 ref 1 00000100:00100000:3.0:1789762879.999331:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403397953445 time 403297953445 state 0x1 ref 1 00000100:00100000:3.0:1789762880.094128:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403397953445 time 403398024538 state 0x0 ref 1 00000100:00100000:3.0:1789762880.094153:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015083904:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762880.094191:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fee lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.094205:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.094210:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.094214:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.094225:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.097178:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fee lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.097191:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fee lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.097198:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.097204:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762880.097208:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fee lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.097214:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fee lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.097229:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fee lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.097234:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04fee lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762880.097286:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015083904:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3140us (98044us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762880.097294:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1060 00000100:00100000:3.0:1789762880.097302:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403397953445 time 403398024538 state 0x2 ref 0 00000100:00100000:0.0:1789762880.099825:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762880.099858:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015084032 00000100:00100000:3.0:1789762880.099864:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015084032 00000100:00100000:3.0:1789762880.099901:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403397953445 time 403398024538 state 0x0 ref 1 00000100:00100000:3.0:1789762880.099912:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403498024538 time 403398024538 state 0x1 ref 1 00000100:00100000:0.0:1789762880.119255:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022181248:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762880.119288:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81ea5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022181376:0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.119303:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81ea4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022181632:0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.119309:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81ea7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022181504:0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.119351:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762880.119359:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762880.119365:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762880.119384:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022181376 00000100:00100000:3.0:1789762880.119390:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022181376 00000100:00100000:1.0:1789762880.119409:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762880.119413:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81ea4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022181888:0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.119421:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81ea4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022181760:0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.119437:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 331 00000100:00100000:2.0:1789762880.119448:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762880.119451:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022181376:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.119458:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762880.119465:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022181632 00000100:00100000:0.0:1789762880.119474:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022181632 00000100:00100000:2.0:1789762880.119492:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022181888 00000100:00100000:1.0:1789762880.119509:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022181760 00000100:00100000:2.0:1789762880.119510:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022181888 00000100:00100000:3.0:1789762880.119511:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022181376:12345-0@lo:400:kworker.0 Request processed in 65us (161us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762880.119521:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 331 00000100:00100000:1.0:1789762880.119524:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022181760 00000100:00100000:0.0:1789762880.119543:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 315 00000100:00100000:2.0:1789762880.119558:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:0.0:1789762880.119560:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022181632:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.119571:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762880.119572:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:2.0:1789762880.119573:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb25880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022181888:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.119588:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022181760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.119596:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81ea5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022182016:0@lo:400:kworker.0 00000100:00100000:0.0:1789762880.119622:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022181632:12345-0@lo:400:kworker.0 Request processed in 70us (258us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762880.119630:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 315 00000100:00100000:2.0:1789762880.119634:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb25880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022181888:12345-0@lo:400:kworker.0 Request processed in 66us (186us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762880.119643:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762880.119645:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022181760:12345-0@lo:400:kworker.0 Request processed in 59us (185us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762880.119653:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00100000:1.0:1789762880.119657:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:0.0:1789762880.119660:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022181504 00000100:00100000:0.0:1789762880.119665:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022181504 00000100:00100000:1.0:1789762880.119684:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022182016 00000100:00100000:1.0:1789762880.119688:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022182016 00000100:00100000:0.0:1789762880.119703:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:0.0:1789762880.119718:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022181504:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.119723:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 158 00000100:00100000:3.0:1789762880.119730:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81ea4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022181888:0@lo:400:kworker.0 00000100:00100000:1.0:1789762880.119736:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022182016:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762880.119749:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81ea5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022181376:0@lo:400:kworker.0 00000100:00080000:2.0:1789762880.119775:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762880.119781:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762880.119798:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022181504:12345-0@lo:400:kworker.0 Request processed in 84us (230us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762880.119804:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81ea7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022181504:0@lo:400:kworker.0 00000100:00100000:0.0:1789762880.119806:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00000100:00100000:1.0:1789762880.119813:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022182016:12345-0@lo:400:kworker.0 Request processed in 80us (173us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762880.119819:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022181248 00000100:00100000:1.0:1789762880.119824:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 158 00000100:00100000:2.0:1789762880.119826:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022181248 00000100:00100000:3.0:1789762880.119833:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81ea5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022182016:0@lo:400:kworker.0 00000100:00100000:0.0:1789762880.119864:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81ea4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022181760:0@lo:400:kworker.0 00000100:00100000:2.0:1789762880.119868:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 132 00000100:00100000:2.0:1789762880.119879:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022181248:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762880.119886:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81ea4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022181632:0@lo:400:kworker.0 00002000:00000024:2.0:1789762880.119903:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762880.119939:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb17b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022181248:12345-0@lo:13:osp-pre-1-0.0 Request processed in 64us (589us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762880.119945:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 132 00000100:00100000:0.0:1789762880.119960:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb25f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022181248:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762880.194214:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403498024538 time 403498109217 state 0x0 ref 1 00000100:00100000:2.0:1789762880.194239:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015084032:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762880.194279:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b04ff5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.194297:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762880.194302:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762880.194309:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b04ff5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.194340:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b04ff5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.196809:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b04ff5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.196818:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b04ff5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.196825:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b04ff5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.196830:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a08000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762880.196833:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b04ff5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.196840:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b04ff5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.196853:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b04ff5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.196857:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000650820a/0x416193e012b04ff5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762880.196914:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015084032:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2685us (97091us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762880.196921:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1061 00000100:00100000:2.0:1789762880.196928:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403498024538 time 403498109217 state 0x2 ref 0 00000100:00100000:2.0:1789762880.198755:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762880.198815:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015084160 00000100:00100000:1.0:1789762880.198824:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015084160 00000100:00100000:1.0:1789762880.198860:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403498024538 time 403498109217 state 0x0 ref 1 00000100:00100000:1.0:1789762880.198872:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403598109217 time 403498109217 state 0x1 ref 1 00000100:00100000:3.0:1789762880.294284:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403598109217 time 403598175755 state 0x0 ref 1 00000100:00100000:3.0:1789762880.294314:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb24380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015084160:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762880.294351:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04ffc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.294366:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.294370:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.294375:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04ffc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.294386:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04ffc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.298127:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04ffc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.298143:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04ffc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.298154:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04ffc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.298164:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762880.298169:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04ffc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.298179:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04ffc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.298201:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04ffc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.298210:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b04ffc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762880.298284:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb24380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015084160:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3978us (99527us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762880.298293:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1062 00000100:00100000:3.0:1789762880.298302:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403598109217 time 403598175755 state 0x2 ref 0 00000100:00100000:1.0:1789762880.300365:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762880.300403:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015084288 00000100:00100000:3.0:1789762880.300413:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015084288 00000100:00100000:3.0:1789762880.300452:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403598109217 time 403598175755 state 0x0 ref 1 00000100:00100000:3.0:1789762880.300462:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403698175755 time 403598175755 state 0x1 ref 1 00000100:00100000:3.0:1789762880.394337:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403698175755 time 403698233283 state 0x0 ref 1 00000100:00100000:3.0:1789762880.394358:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015084288:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762880.394393:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05003 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.394407:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.394410:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.394415:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05003 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.394426:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05003 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.397682:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05003 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.397692:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05003 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.397698:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05003 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.397703:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762880.397706:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05003 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.397713:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05003 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.397726:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05003 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.397732:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05003 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762880.397791:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015084288:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3438us (97433us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762880.397797:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1063 00000100:00100000:3.0:1789762880.397804:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403698175755 time 403698233283 state 0x2 ref 0 00000100:00100000:1.0:1789762880.399704:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762880.399738:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015084416 00000100:00100000:3.0:1789762880.399746:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015084416 00000100:00100000:3.0:1789762880.399804:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403698175755 time 403698233283 state 0x0 ref 1 00000100:00100000:3.0:1789762880.399820:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403798233283 time 403698233283 state 0x1 ref 1 00010000:00010000:0.0:1789762880.439221:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762880.494437:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403798233283 time 403798329302 state 0x0 ref 1 00000100:00100000:2.0:1789762880.494466:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015084416:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762880.494510:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0500a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.494524:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762880.494527:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762880.494532:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0500a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.494542:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0500a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.497649:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0500a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.497663:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0500a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.497685:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0500a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.497695:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9f714200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762880.497701:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0500a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.497711:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0500a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.497728:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0500a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.497734:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0500a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762880.497797:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015084416:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3342us (98093us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762880.497804:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1064 00000100:00100000:2.0:1789762880.497812:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403798233283 time 403798329302 state 0x2 ref 0 00000100:00100000:2.0:1789762880.499694:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762880.499735:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015084544 00000100:00100000:1.0:1789762880.499743:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015084544 00000100:00100000:1.0:1789762880.499797:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403798233283 time 403798329302 state 0x0 ref 1 00000100:00100000:1.0:1789762880.499811:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403898329302 time 403798329302 state 0x1 ref 1 00000100:00100000:3.0:1789762880.594582:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403898329302 time 403898475449 state 0x0 ref 1 00000100:00100000:3.0:1789762880.594607:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015084544:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762880.594643:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05011 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.594656:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.594660:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.594666:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05011 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.594681:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05011 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.597660:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05011 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.597684:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05011 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.597696:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05011 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.597704:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762880.597710:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05011 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.597733:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05011 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.597771:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05011 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.597780:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05011 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762880.597875:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015084544:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3271us (98177us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762880.597890:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1065 00000100:00100000:3.0:1789762880.597903:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403898329302 time 403898475449 state 0x2 ref 0 00000100:00100000:2.0:1789762880.599982:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762880.600025:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015084672 00000100:00100000:3.0:1789762880.600036:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015084672 00000100:00100000:3.0:1789762880.600106:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403898329302 time 403898475449 state 0x0 ref 1 00000100:00100000:3.0:1789762880.600120:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403998475449 time 403898475449 state 0x1 ref 1 00000100:00100000:3.0:1789762880.694679:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403998475449 time 403998570237 state 0x0 ref 1 00000100:00100000:3.0:1789762880.694710:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015084672:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762880.694770:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05018 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.694789:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.694793:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.694799:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05018 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.694812:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05018 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.698299:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05018 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.698311:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05018 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.698320:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05018 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.698326:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762880.698332:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05018 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.698342:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05018 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.698369:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05018 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.698379:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05018 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762880.698445:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015084672:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3748us (98467us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762880.698452:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1066 00000100:00100000:3.0:1789762880.698462:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403998475449 time 403998570237 state 0x2 ref 0 00000100:00100000:1.0:1789762880.700396:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762880.700435:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015084800 00000100:00100000:3.0:1789762880.700446:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015084800 00000100:00100000:3.0:1789762880.700486:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 403998475449 time 403998570237 state 0x0 ref 1 00000100:00100000:3.0:1789762880.700497:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404098570237 time 403998570237 state 0x1 ref 1 00000100:00100000:2.0:1789762880.794801:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404098570237 time 404098696574 state 0x0 ref 1 00000100:00100000:2.0:1789762880.794844:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015084800:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762880.794881:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0501f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.794894:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762880.794898:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762880.794903:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0501f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.794914:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0501f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.798122:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0501f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.798132:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0501f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.798138:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0501f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.798143:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9f714200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762880.798147:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0501f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.798153:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0501f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.798174:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0501f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762880.798180:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096911fae/0x416193e012b0501f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762880.798232:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015084800:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3395us (97843us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762880.798238:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1067 00000100:00100000:2.0:1789762880.798245:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404098570237 time 404098696574 state 0x2 ref 0 00000100:00100000:1.0:1789762880.800342:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762880.800416:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015084928 00000100:00100000:2.0:1789762880.800425:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015084928 00000100:00100000:2.0:1789762880.800461:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404098570237 time 404098696574 state 0x0 ref 1 00000100:00100000:2.0:1789762880.800472:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404198696574 time 404098696574 state 0x1 ref 1 00000100:00100000:3.0:1789762880.894927:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404198696574 time 404198815764 state 0x0 ref 1 00000100:00100000:3.0:1789762880.894962:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015084928:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762880.895013:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05026 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.895032:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.895036:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.895041:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05026 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.895055:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05026 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.898473:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05026 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.898485:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05026 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.898493:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05026 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.898498:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762880.898503:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05026 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.898511:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05026 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.898530:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05026 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.898537:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05026 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762880.898601:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015084928:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3655us (98266us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762880.898609:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1068 00000100:00100000:3.0:1789762880.898617:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404198696574 time 404198815764 state 0x2 ref 0 00000100:00100000:1.0:1789762880.900650:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762880.900685:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015085056 00000100:00100000:3.0:1789762880.900695:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015085056 00000100:00100000:3.0:1789762880.900734:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404198696574 time 404198815764 state 0x0 ref 1 00000100:00100000:3.0:1789762880.900749:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404298815764 time 404198815764 state 0x1 ref 1 00000100:00100000:3.0:1789762880.995002:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404298815764 time 404298885412 state 0x0 ref 1 00000100:00100000:3.0:1789762880.995037:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015085056:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762880.995130:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0502d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.995152:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.995156:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762880.995163:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0502d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.995176:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0502d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.998554:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0502d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.998564:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0502d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.998570:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0502d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.998580:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762880.998584:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0502d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.998591:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0502d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.998607:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0502d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762880.998612:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0502d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762880.998671:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015085056:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3650us (98022us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762880.998678:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1069 00000100:00100000:3.0:1789762880.998686:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404298815764 time 404298885412 state 0x2 ref 0 00000100:00100000:1.0:1789762881.000528:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762881.000562:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015085184 00000100:00100000:3.0:1789762881.000571:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015085184 00000100:00080000:2.0:1789762881.000618:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762881.000626:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404298815764 time 404298885412 state 0x0 ref 1 00000100:00100000:3.0:1789762881.000639:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404398885412 time 404298885412 state 0x1 ref 1 00000100:00100000:2.0:1789762881.095081:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404398885412 time 404398971592 state 0x0 ref 1 00000100:00100000:2.0:1789762881.095127:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015085184:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762881.095175:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05034 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762881.095213:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762881.095217:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762881.095223:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05034 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762881.095239:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05034 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.098770:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05034 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.098783:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05034 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.098791:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05034 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.098797:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762881.098801:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05034 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.098811:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05034 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.098828:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05034 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.098833:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05034 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762881.098892:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015085184:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3777us (98365us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762881.098900:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1070 00000100:00100000:1.0:1789762881.098909:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404398885412 time 404398971592 state 0x2 ref 0 00000100:00100000:1.0:1789762881.100654:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762881.100706:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015085312 00000100:00100000:2.0:1789762881.100718:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015085312 00000100:00100000:2.0:1789762881.100774:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404398885412 time 404398971592 state 0x0 ref 1 00000100:00100000:2.0:1789762881.100791:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404498971592 time 404398971592 state 0x1 ref 1 00000100:00100000:3.0:1789762881.195173:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404498971592 time 404499062364 state 0x0 ref 1 00000100:00100000:3.0:1789762881.195205:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015085312:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762881.195253:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0503b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.195271:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762881.195276:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762881.195282:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0503b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.195295:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0503b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.198770:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0503b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.198788:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0503b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.198800:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0503b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.198810:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762881.198816:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0503b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.198827:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0503b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.198850:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0503b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.198857:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b0503b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762881.198928:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015085312:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3734us (98271us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762881.198939:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1071 00000100:00100000:3.0:1789762881.198952:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404498971592 time 404499062364 state 0x2 ref 0 00000100:00100000:1.0:1789762881.200932:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762881.200972:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015085440 00000100:00100000:3.0:1789762881.200983:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015085440 00000100:00100000:3.0:1789762881.201036:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404498971592 time 404499062364 state 0x0 ref 1 00000100:00100000:3.0:1789762881.201050:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404599062364 time 404499062364 state 0x1 ref 1 00000100:00100000:3.0:1789762881.295249:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404599062364 time 404599139419 state 0x0 ref 1 00000100:00100000:3.0:1789762881.295279:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015085440:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762881.295331:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05042 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.295350:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762881.295355:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762881.295361:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05042 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.295374:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05042 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.298653:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05042 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.298662:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05042 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.298669:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05042 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.298674:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762881.298678:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05042 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.298685:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05042 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.298701:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05042 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.298706:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05042 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762881.298782:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015085440:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3514us (97850us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762881.298793:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1072 00000100:00100000:3.0:1789762881.298803:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404599062364 time 404599139419 state 0x2 ref 0 00000100:00100000:2.0:1789762881.300550:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762881.300591:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015085568 00000100:00100000:3.0:1789762881.300602:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015085568 00000100:00100000:3.0:1789762881.300651:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404599062364 time 404599139419 state 0x0 ref 1 00000100:00100000:3.0:1789762881.300671:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404699139419 time 404599139419 state 0x1 ref 1 00000100:00100000:1.0:1789762881.395383:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404699139419 time 404699268952 state 0x0 ref 1 00000100:00100000:1.0:1789762881.395417:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015085568:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762881.395470:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05049 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.395491:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762881.395496:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762881.395505:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05049 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.395522:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05049 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.398568:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05049 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.398581:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05049 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.398592:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05049 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.398601:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762881.398607:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05049 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.398618:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05049 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.398639:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05049 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.398648:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05049 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762881.398717:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015085568:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3314us (98168us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762881.398728:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1073 00000100:00100000:1.0:1789762881.398740:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404699139419 time 404699268952 state 0x2 ref 0 00000100:00100000:2.0:1789762881.400475:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762881.400515:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015085696 00000100:00100000:1.0:1789762881.400524:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015085696 00000100:00100000:1.0:1789762881.400573:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404699139419 time 404699268952 state 0x0 ref 1 00000100:00100000:1.0:1789762881.400589:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404799268952 time 404699268952 state 0x1 ref 1 00010000:00010000:2.0:1789762881.463239:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762881.495471:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404799268952 time 404799364850 state 0x0 ref 1 00000100:00100000:3.0:1789762881.495507:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb26a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015085696:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762881.495557:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05050 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.495576:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762881.495580:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762881.495586:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05050 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.495599:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05050 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.499114:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05050 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.499139:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05050 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.499150:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05050 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.499160:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762881.499166:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05050 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.499176:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05050 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.499216:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05050 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.499224:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077f9d1fb/0x416193e012b05050 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762881.499333:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb26a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015085696:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3835us (98852us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762881.499346:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1074 00000100:00100000:0.0:1789762881.499360:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404799268952 time 404799364850 state 0x2 ref 0 00000100:00100000:2.0:1789762881.501665:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762881.501701:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015085824 00000100:00100000:3.0:1789762881.501712:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015085824 00000100:00100000:3.0:1789762881.501777:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404799268952 time 404799364850 state 0x0 ref 1 00000100:00100000:3.0:1789762881.501790:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404899364850 time 404799364850 state 0x1 ref 1 00000100:00100000:3.0:1789762881.595527:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404899364850 time 404899422624 state 0x0 ref 1 00000100:00100000:3.0:1789762881.595555:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015085824:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762881.595592:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05057 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.595606:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762881.595609:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762881.595614:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05057 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.595624:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05057 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.598678:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05057 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.598688:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05057 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.598695:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05057 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.598700:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762881.598703:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05057 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.598709:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05057 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.598722:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05057 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.598727:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05057 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762881.598786:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015085824:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3243us (97123us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762881.598793:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1075 00000100:00100000:3.0:1789762881.598800:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404899364850 time 404899422624 state 0x2 ref 0 00000100:00100000:1.0:1789762881.600660:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762881.600696:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015085952 00000100:00100000:3.0:1789762881.600704:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015085952 00000100:00100000:3.0:1789762881.600736:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404899364850 time 404899422624 state 0x0 ref 1 00000100:00100000:3.0:1789762881.600760:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404999422624 time 404899422624 state 0x1 ref 1 00000100:00100000:1.0:1789762881.695593:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404999422624 time 404999489540 state 0x0 ref 1 00000100:00100000:1.0:1789762881.695621:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015085952:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762881.695659:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000784cdf1b/0x416193e012b0505e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.695672:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762881.695676:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762881.695680:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000784cdf1b/0x416193e012b0505e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.695690:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000784cdf1b/0x416193e012b0505e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.698561:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000784cdf1b/0x416193e012b0505e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.698572:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000784cdf1b/0x416193e012b0505e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.698579:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000784cdf1b/0x416193e012b0505e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.698584:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8aa41a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762881.698588:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000784cdf1b/0x416193e012b0505e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.698594:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000784cdf1b/0x416193e012b0505e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.698609:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000784cdf1b/0x416193e012b0505e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.698615:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000784cdf1b/0x416193e012b0505e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762881.698668:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015085952:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3058us (98008us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762881.698677:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1076 00000100:00100000:1.0:1789762881.698685:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404999422624 time 404999489540 state 0x2 ref 0 00000100:00100000:1.0:1789762881.700841:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762881.700888:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015086080 00000100:00100000:3.0:1789762881.700897:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015086080 00000100:00100000:3.0:1789762881.700936:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 404999422624 time 404999489540 state 0x0 ref 1 00000100:00100000:3.0:1789762881.700947:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405099489540 time 404999489540 state 0x1 ref 1 00000100:00100000:0.0:1789762881.795677:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405099489540 time 405099573335 state 0x0 ref 1 00000100:00100000:0.0:1789762881.795702:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015086080:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:0.0:1789762881.795739:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b05065 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.795764:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762881.795768:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762881.795773:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b05065 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.795788:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b05065 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.799194:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b05065 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.799205:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b05065 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.799212:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b05065 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.799217:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d864ca000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762881.799220:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b05065 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.799226:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b05065 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.799238:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b05065 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762881.799243:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b05065 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762881.799293:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015086080:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3601us (98453us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762881.799300:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1077 00000100:00100000:0.0:1789762881.799314:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405099489540 time 405099573335 state 0x2 ref 0 00000100:00100000:1.0:1789762881.803252:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762881.803292:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015086208 00000100:00100000:3.0:1789762881.803301:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015086208 00000100:00100000:3.0:1789762881.803347:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405099489540 time 405099573335 state 0x0 ref 1 00000100:00100000:3.0:1789762881.803359:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405199573335 time 405099573335 state 0x1 ref 1 00000100:00100000:3.0:1789762881.895771:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405199573335 time 405199649830 state 0x0 ref 1 00000100:00100000:3.0:1789762881.895800:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015086208:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762881.895841:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0506c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.895855:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762881.895858:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762881.895863:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0506c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.895879:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0506c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.898943:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0506c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.898953:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0506c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.898959:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0506c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.898964:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762881.898968:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0506c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.898975:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0506c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.898988:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0506c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762881.898994:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0506c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762881.899043:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015086208:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3255us (95797us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762881.899050:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1078 00000100:00100000:3.0:1789762881.899058:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405199573335 time 405199649830 state 0x2 ref 0 00000100:00100000:1.0:1789762881.901064:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762881.901111:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015086336 00000100:00100000:3.0:1789762881.901118:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015086336 00000100:00100000:3.0:1789762881.901150:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405199573335 time 405199649830 state 0x0 ref 1 00000100:00100000:3.0:1789762881.901159:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405299649830 time 405199649830 state 0x1 ref 1 00000100:00100000:1.0:1789762881.995861:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405299649830 time 405299756457 state 0x0 ref 1 00000100:00100000:1.0:1789762881.995886:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015086336:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762881.995922:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b05073 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.995935:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762881.995939:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762881.995944:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b05073 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762881.995955:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b05073 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762881.998715:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b05073 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762881.998726:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b05073 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762881.998733:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b05073 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762881.998762:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762881.998766:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b05073 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762881.998773:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b05073 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762881.998786:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b05073 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762881.998791:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f608de43/0x416193e012b05073 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762881.998839:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015086336:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2963us (97776us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762881.998847:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1079 00000100:00100000:2.0:1789762881.998855:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405299649830 time 405299756457 state 0x2 ref 0 00000100:00100000:2.0:1789762882.000999:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762882.001044:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015086464 00000100:00100000:3.0:1789762882.001052:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015086464 00000100:00100000:3.0:1789762882.001106:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405299649830 time 405299756457 state 0x0 ref 1 00000100:00100000:3.0:1789762882.001118:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405399756457 time 405299756457 state 0x1 ref 1 00000100:00100000:0.0:1789762882.095899:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405399756457 time 405399793977 state 0x0 ref 1 00000100:00100000:0.0:1789762882.095923:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015086464:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:0.0:1789762882.095960:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b0507a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762882.095975:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762882.095980:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762882.095987:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b0507a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762882.096004:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b0507a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.102236:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b0507a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.102249:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b0507a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.102257:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b0507a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.102265:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d864ca000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762882.102270:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b0507a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.102276:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b0507a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.102294:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b0507a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.102299:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eaa213e2/0x416193e012b0507a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762882.102356:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015086464:12345-192.168.203.48@tcp:3:dd.0 Request processed in 6438us (101355us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762882.102365:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1080 00000100:00100000:1.0:1789762882.102374:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405399756457 time 405399793977 state 0x2 ref 0 00000100:00100000:1.0:1789762882.104301:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762882.104343:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015086592 00000100:00100000:0.0:1789762882.104352:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015086592 00000100:00100000:0.0:1789762882.104394:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405399756457 time 405399793977 state 0x0 ref 1 00000100:00100000:0.0:1789762882.104410:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405499793977 time 405399793977 state 0x1 ref 1 00000100:00100000:3.0:1789762882.195987:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405499793977 time 405499884434 state 0x0 ref 1 00000100:00100000:3.0:1789762882.196012:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015086592:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762882.196049:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05081 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.196063:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762882.196067:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762882.196072:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05081 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.196082:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05081 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.199305:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05081 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.199315:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05081 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.199322:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05081 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.199327:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762882.199330:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05081 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.199336:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05081 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.199348:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05081 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.199354:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05081 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762882.199400:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015086592:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3397us (95102us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762882.199407:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1081 00000100:00100000:3.0:1789762882.199415:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405499793977 time 405499884434 state 0x2 ref 0 00000100:00100000:1.0:1789762882.201152:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762882.201190:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015086720 00000100:00100000:3.0:1789762882.201198:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015086720 00000100:00100000:3.0:1789762882.201232:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405499793977 time 405499884434 state 0x0 ref 1 00000100:00100000:3.0:1789762882.201243:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405599884434 time 405499884434 state 0x1 ref 1 00000100:00100000:2.0:1789762882.296134:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405599884434 time 405600030139 state 0x0 ref 1 00000100:00100000:2.0:1789762882.296158:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015086720:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762882.296195:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b05088 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.296223:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762882.296227:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762882.296232:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b05088 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.296243:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b05088 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.299376:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b05088 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.299386:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b05088 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.299394:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b05088 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.299399:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762882.299402:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b05088 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.299409:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b05088 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.299424:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b05088 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.299439:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b05088 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762882.299488:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015086720:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3338us (98337us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762882.299495:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1082 00000100:00100000:2.0:1789762882.299502:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405599884434 time 405600030139 state 0x2 ref 0 00000100:00100000:1.0:1789762882.301234:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762882.301282:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015086848 00000100:00100000:3.0:1789762882.301291:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015086848 00000100:00100000:3.0:1789762882.301333:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405599884434 time 405600030139 state 0x0 ref 1 00000100:00100000:3.0:1789762882.301344:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405700030139 time 405600030139 state 0x1 ref 1 00000100:00100000:1.0:1789762882.396215:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405700030139 time 405700111112 state 0x0 ref 1 00000100:00100000:1.0:1789762882.396238:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015086848:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762882.396273:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0508f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.396287:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762882.396291:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762882.396296:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0508f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.396306:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0508f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.399408:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0508f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.399422:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0508f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.399432:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0508f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.399442:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762882.399447:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0508f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.399456:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0508f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.399478:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0508f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.399487:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0508f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762882.399548:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015086848:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3318us (98314us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762882.399555:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1083 00000100:00100000:1.0:1789762882.399562:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405700030139 time 405700111112 state 0x2 ref 0 00000100:00100000:1.0:1789762882.401431:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762882.401474:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015086976 00000100:00100000:3.0:1789762882.401482:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015086976 00000100:00100000:3.0:1789762882.401518:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405700030139 time 405700111112 state 0x0 ref 1 00000100:00100000:3.0:1789762882.401528:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405800111112 time 405700111112 state 0x1 ref 1 00010000:00010000:3.0:1789762882.487260:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762882.496274:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405800111112 time 405800173451 state 0x0 ref 1 00000100:00100000:3.0:1789762882.496297:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015086976:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762882.496331:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05096 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.496345:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762882.496348:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762882.496353:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05096 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.496363:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05096 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.499147:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05096 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.499157:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05096 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.499165:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05096 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.499170:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762882.499174:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05096 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.499185:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05096 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.499200:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05096 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.499205:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05096 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762882.499252:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015086976:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2964us (97822us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762882.499259:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1084 00000100:00100000:3.0:1789762882.499266:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405800111112 time 405800173451 state 0x2 ref 0 00000100:00100000:0.0:1789762882.501195:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762882.501231:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015087104 00000100:00100000:3.0:1789762882.501239:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015087104 00000100:00100000:3.0:1789762882.501272:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405800111112 time 405800173451 state 0x0 ref 1 00000100:00100000:3.0:1789762882.501283:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405900173451 time 405800173451 state 0x1 ref 1 00000100:00100000:2.0:1789762882.596453:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405900173451 time 405900324809 state 0x0 ref 1 00000100:00100000:2.0:1789762882.596483:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015087104:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762882.596547:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0509d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.596570:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762882.596576:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762882.596585:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0509d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.596603:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0509d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.600349:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0509d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.600394:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0509d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.600413:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0509d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.600440:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762882.600445:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0509d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.600455:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0509d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.600473:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0509d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.600479:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b0509d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762882.600591:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015087104:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4103us (99380us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762882.600598:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1085 00000100:00100000:2.0:1789762882.600604:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405900173451 time 405900324809 state 0x2 ref 0 00000100:00100000:2.0:1789762882.602846:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762882.602921:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015087232 00000100:00100000:3.0:1789762882.602935:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015087232 00000100:00100000:3.0:1789762882.602994:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 405900173451 time 405900324809 state 0x0 ref 1 00000100:00100000:3.0:1789762882.603013:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406000324809 time 405900324809 state 0x1 ref 1 00000100:00100000:1.0:1789762882.696593:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406000324809 time 406000480574 state 0x0 ref 1 00000100:00100000:1.0:1789762882.696632:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015087232:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762882.696685:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b20687d/0x416193e012b050a4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.696709:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762882.696716:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762882.696724:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b20687d/0x416193e012b050a4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762882.696767:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b20687d/0x416193e012b050a4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762882.702338:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b20687d/0x416193e012b050a4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762882.702380:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b20687d/0x416193e012b050a4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762882.702401:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b20687d/0x416193e012b050a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762882.702417:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebad3e600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762882.702428:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b20687d/0x416193e012b050a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762882.702447:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b20687d/0x416193e012b050a4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762882.702496:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b20687d/0x416193e012b050a4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762882.702509:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b20687d/0x416193e012b050a4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762882.702594:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015087232:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5975us (99743us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762882.702607:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1086 00000100:00100000:0.0:1789762882.702621:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406000324809 time 406000480574 state 0x2 ref 0 00000100:00100000:2.0:1789762882.706450:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762882.706523:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015087360 00000100:00100000:3.0:1789762882.706537:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015087360 00000100:00100000:3.0:1789762882.706598:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406000324809 time 406000480574 state 0x0 ref 1 00000100:00100000:3.0:1789762882.706617:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406100480574 time 406000480574 state 0x1 ref 1 00000100:00100000:3.0:1789762882.796703:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406100480574 time 406100589709 state 0x0 ref 1 00000100:00100000:3.0:1789762882.796768:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb24e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015087360:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762882.796825:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ab lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.796849:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762882.796855:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762882.796863:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ab lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.796882:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ab lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.801812:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ab lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.801833:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ab lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.801853:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ab lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.801863:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762882.801869:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ab lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.801880:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ab lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.801901:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ab lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762882.801911:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ab lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762882.801998:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb24e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015087360:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5268us (95550us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762882.802010:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1087 00000100:00100000:3.0:1789762882.802023:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406100480574 time 406100589709 state 0x2 ref 0 00000100:00100000:2.0:1789762882.805866:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762882.805931:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015087488 00000100:00100000:3.0:1789762882.805945:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015087488 00000100:00100000:3.0:1789762882.806002:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406100480574 time 406100589709 state 0x0 ref 1 00000100:00100000:3.0:1789762882.806027:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406200589709 time 406100589709 state 0x1 ref 1 00000100:00100000:2.0:1789762882.896904:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406200589709 time 406200791981 state 0x0 ref 1 00000100:00100000:2.0:1789762882.896944:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb24a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015087488:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762882.897015:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b050b2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.897038:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762882.897043:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762882.897051:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b050b2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.897070:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b050b2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.902338:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b050b2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.902359:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b050b2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.902373:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b050b2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.902382:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762882.902388:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b050b2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.902412:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b050b2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.902436:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b050b2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762882.902447:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000086782296/0x416193e012b050b2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762882.902543:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb24a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015087488:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5612us (96678us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762882.902556:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1088 00000100:00100000:2.0:1789762882.902569:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406200589709 time 406200791981 state 0x2 ref 0 00000100:00100000:2.0:1789762882.906073:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762882.906153:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015087616 00000100:00100000:3.0:1789762882.906166:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015087616 00000100:00100000:3.0:1789762882.906226:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406200589709 time 406200791981 state 0x0 ref 1 00000100:00100000:3.0:1789762882.906245:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406300791981 time 406200791981 state 0x1 ref 1 00000100:00100000:0.0:1789762882.997174:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406300791981 time 406301057816 state 0x0 ref 1 00000100:00100000:0.0:1789762882.997232:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015087616:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:0.0:1789762882.997288:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c118925b/0x416193e012b050b9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762882.997311:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762882.997318:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762882.997327:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c118925b/0x416193e012b050b9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762882.997345:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c118925b/0x416193e012b050b9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.003831:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c118925b/0x416193e012b050b9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.003865:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c118925b/0x416193e012b050b9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.003885:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c118925b/0x416193e012b050b9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.003898:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc4df800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762883.003908:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c118925b/0x416193e012b050b9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.003921:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c118925b/0x416193e012b050b9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.003970:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c118925b/0x416193e012b050b9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.003985:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c118925b/0x416193e012b050b9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762883.004164:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015087616:12345-192.168.203.48@tcp:3:dd.0 Request processed in 6936us (98087us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762883.004181:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1089 00000100:00100000:0.0:1789762883.004197:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406300791981 time 406301057816 state 0x2 ref 0 00000100:00100000:1.0:1789762883.008423:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762883.008506:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015087744 00000100:00100000:3.0:1789762883.008520:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015087744 00000100:00100000:3.0:1789762883.008627:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406300791981 time 406301057816 state 0x0 ref 1 00000100:00080000:2.0:1789762883.008641:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762883.008655:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406401057816 time 406301057816 state 0x1 ref 1 00000100:00100000:3.0:1789762883.097312:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406401057816 time 406401195621 state 0x0 ref 1 00000100:00100000:3.0:1789762883.097349:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015087744:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762883.097419:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050c0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.097443:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762883.097449:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762883.097457:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050c0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.097485:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050c0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.103020:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050c0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.103042:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050c0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.103056:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050c0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.103066:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762883.103073:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050c0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.103119:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050c0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.103146:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050c0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.103156:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050c0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762883.103261:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015087744:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5919us (94837us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762883.103282:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1090 00000100:00100000:3.0:1789762883.103303:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406401057816 time 406401195621 state 0x2 ref 0 00000100:00100000:1.0:1789762883.106539:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762883.106612:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015087872 00000100:00100000:3.0:1789762883.106629:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015087872 00000100:00100000:3.0:1789762883.106702:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406401057816 time 406401195621 state 0x0 ref 1 00000100:00100000:3.0:1789762883.106721:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406501195621 time 406401195621 state 0x1 ref 1 00000100:00100000:2.0:1789762883.197486:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406501195621 time 406501370833 state 0x0 ref 1 00000100:00100000:2.0:1789762883.197524:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015087872:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762883.197580:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000059872ac4/0x416193e012b050c7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762883.197619:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762883.197625:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762883.197650:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000059872ac4/0x416193e012b050c7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762883.197669:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000059872ac4/0x416193e012b050c7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762883.203389:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000059872ac4/0x416193e012b050c7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762883.203412:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000059872ac4/0x416193e012b050c7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762883.203424:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000059872ac4/0x416193e012b050c7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762883.203433:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9f75e600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762883.203440:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000059872ac4/0x416193e012b050c7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762883.203451:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000059872ac4/0x416193e012b050c7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762883.203480:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000059872ac4/0x416193e012b050c7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762883.203496:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000059872ac4/0x416193e012b050c7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762883.203631:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015087872:12345-192.168.203.48@tcp:3:dd.0 Request processed in 6116us (97090us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762883.203647:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1091 00000100:00100000:2.0:1789762883.203663:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406501195621 time 406501370833 state 0x2 ref 0 00000100:00100000:2.0:1789762883.206203:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762883.206243:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015088000 00000100:00100000:3.0:1789762883.206259:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015088000 00000100:00100000:3.0:1789762883.206297:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406501195621 time 406501370833 state 0x0 ref 1 00000100:00100000:3.0:1789762883.206308:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406601370833 time 406501370833 state 0x1 ref 1 00000100:00100000:0.0:1789762883.297635:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406601370833 time 406601523332 state 0x0 ref 1 00000100:00100000:0.0:1789762883.297670:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb25500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015088000:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:0.0:1789762883.297723:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b050ce lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.297772:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762883.297778:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762883.297787:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b050ce lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.297805:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b050ce lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.302372:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b050ce lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.302392:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b050ce lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.302406:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b050ce lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.302416:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762883.302422:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b050ce lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.302433:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b050ce lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.302458:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b050ce lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.302467:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b050ce lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762883.302550:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb25500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015088000:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4890us (96342us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762883.302561:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1092 00000100:00100000:0.0:1789762883.302590:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406601370833 time 406601523332 state 0x2 ref 0 00000100:00100000:0.0:1789762883.304725:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762883.304843:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015088128 00000100:00100000:3.0:1789762883.304857:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015088128 00000100:00100000:3.0:1789762883.304916:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406601370833 time 406601523332 state 0x0 ref 1 00000100:00100000:3.0:1789762883.304934:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406701523332 time 406601523332 state 0x1 ref 1 00000100:00100000:3.0:1789762883.397676:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406701523332 time 406701572924 state 0x0 ref 1 00000100:00100000:3.0:1789762883.397697:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe72a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015088128:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762883.397731:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050d5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.397759:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762883.397763:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762883.397772:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050d5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.397784:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050d5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.400862:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050d5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.400870:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050d5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.400876:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050d5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.400881:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762883.400884:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050d5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.400890:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050d5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.400903:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050d5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.400909:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050d5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762883.400954:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe72a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015088128:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3263us (96234us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762883.400960:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1093 00000100:00100000:3.0:1789762883.400967:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406701523332 time 406701572924 state 0x2 ref 0 00000100:00100000:1.0:1789762883.402708:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762883.402764:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015088256 00000100:00100000:2.0:1789762883.402772:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015088256 00000100:00100000:2.0:1789762883.402808:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406701523332 time 406701572924 state 0x0 ref 1 00000100:00100000:2.0:1789762883.402819:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406801572924 time 406701572924 state 0x1 ref 1 00000100:00100000:2.0:1789762883.497724:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406801572924 time 406801622146 state 0x0 ref 1 00000100:00100000:2.0:1789762883.497760:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015088256:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762883.497795:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b050dc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762883.497809:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762883.497812:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762883.497817:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b050dc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762883.497827:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b050dc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.500435:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b050dc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.500447:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b050dc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.500454:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b050dc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.500460:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762883.500464:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b050dc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.500470:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b050dc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.500484:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b050dc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.500489:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b050dc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762883.500539:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015088256:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2800us (97832us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762883.500547:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1094 00000100:00100000:1.0:1789762883.500554:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406801572924 time 406801622146 state 0x2 ref 0 00000100:00100000:1.0:1789762883.502445:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762883.502484:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015088384 00000100:00100000:3.0:1789762883.502491:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015088384 00000100:00100000:3.0:1789762883.502534:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406801572924 time 406801622146 state 0x0 ref 1 00000100:00100000:3.0:1789762883.502550:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406901622146 time 406801622146 state 0x1 ref 1 00010000:00010000:0.0:1789762883.511203:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762883.597817:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406901622146 time 406901714059 state 0x0 ref 1 00000100:00100000:0.0:1789762883.597839:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015088384:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:0.0:1789762883.597876:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b050e3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.597890:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762883.597894:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762883.597899:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b050e3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762883.597910:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b050e3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.601257:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b050e3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.601273:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b050e3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.601284:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b050e3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.601290:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aae00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762883.601293:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b050e3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.601300:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b050e3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.601314:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b050e3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.601324:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b050e3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762883.601379:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015088384:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3545us (98934us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762883.601388:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1095 00000100:00100000:1.0:1789762883.601399:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406901622146 time 406901714059 state 0x2 ref 0 00000100:00100000:1.0:1789762883.603951:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762883.603990:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015088512 00000100:00100000:3.0:1789762883.603998:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015088512 00000100:00100000:3.0:1789762883.604035:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 406901622146 time 406901714059 state 0x0 ref 1 00000100:00100000:3.0:1789762883.604045:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407001714059 time 406901714059 state 0x1 ref 1 00000100:00100000:3.0:1789762883.697889:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407001714059 time 407001784909 state 0x0 ref 1 00000100:00100000:3.0:1789762883.697917:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015088512:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762883.697953:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ea lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.697966:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762883.697969:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762883.697974:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.697986:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.701060:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.701071:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ea lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.701079:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.701102:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762883.701106:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.701114:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.701130:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ea lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.701136:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ea lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762883.701183:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015088512:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3275us (97233us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762883.701191:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1096 00000100:00100000:3.0:1789762883.701198:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407001714059 time 407001784909 state 0x2 ref 0 00000100:00100000:2.0:1789762883.702903:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762883.702954:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015088640 00000100:00100000:1.0:1789762883.702962:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015088640 00000100:00100000:1.0:1789762883.703001:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407001714059 time 407001784909 state 0x0 ref 1 00000100:00100000:1.0:1789762883.703012:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407101784909 time 407001784909 state 0x1 ref 1 00000100:00100000:1.0:1789762883.797946:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407101784909 time 407101841344 state 0x0 ref 1 00000100:00100000:1.0:1789762883.797971:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015088640:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762883.798009:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.798022:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762883.798026:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762883.798030:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.798042:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.801654:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.801666:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.801673:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.801679:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762883.801683:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.801689:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.801703:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.801708:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762883.801771:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015088640:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3810us (98869us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762883.801778:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1097 00000100:00100000:1.0:1789762883.801786:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407101784909 time 407101841344 state 0x2 ref 0 00000100:00100000:0.0:1789762883.804159:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762883.804200:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015088768 00000100:00100000:3.0:1789762883.804208:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015088768 00000100:00100000:3.0:1789762883.804248:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407101784909 time 407101841344 state 0x0 ref 1 00000100:00100000:3.0:1789762883.804263:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407201841344 time 407101841344 state 0x1 ref 1 00000100:00100000:1.0:1789762883.898025:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407201841344 time 407201921273 state 0x0 ref 1 00000100:00100000:1.0:1789762883.898051:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe72680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015088768:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762883.898106:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.898120:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762883.898124:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762883.898128:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.898139:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.900948:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.900959:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.900970:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.900977:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762883.900980:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.900987:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.901001:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762883.901007:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b050f8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762883.901052:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe72680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015088768:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3011us (96893us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762883.901060:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1098 00000100:00100000:1.0:1789762883.901068:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407201841344 time 407201921273 state 0x2 ref 0 00000100:00100000:0.0:1789762883.902987:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762883.903036:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015088896 00000100:00100000:3.0:1789762883.903046:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015088896 00000100:00100000:3.0:1789762883.903121:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407201841344 time 407201921273 state 0x0 ref 1 00000100:00100000:3.0:1789762883.903138:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407301921273 time 407201921273 state 0x1 ref 1 00000100:00100000:3.0:1789762883.998078:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407301921273 time 407301973851 state 0x0 ref 1 00000100:00100000:3.0:1789762883.998127:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015088896:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762883.998163:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ff lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.998184:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762883.998190:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762883.998197:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762883.998212:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.000929:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ff lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.000938:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ff lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.000944:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.000949:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762884.000953:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ff lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.000959:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ff lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.000972:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ff lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.000977:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b050ff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762884.001021:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015088896:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2904us (98035us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762884.001028:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1099 00000100:00100000:3.0:1789762884.001036:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407301921273 time 407301973851 state 0x2 ref 0 00000100:00100000:0.0:1789762884.002811:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762884.002861:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015089024 00000100:00100000:1.0:1789762884.002869:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015089024 00000100:00100000:1.0:1789762884.002906:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407301921273 time 407301973851 state 0x0 ref 1 00000100:00100000:1.0:1789762884.002919:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407401973851 time 407301973851 state 0x1 ref 1 00000100:00100000:1.0:1789762884.098125:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407401973851 time 407402022564 state 0x0 ref 1 00000100:00100000:1.0:1789762884.098145:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe73100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015089024:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762884.098179:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05106 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.098193:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.098197:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.098201:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05106 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.098212:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05106 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.101322:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05106 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.101333:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05106 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.101341:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05106 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.101351:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762884.101355:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05106 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.101368:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05106 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.101382:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05106 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.101387:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05106 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762884.101434:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe73100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015089024:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3294us (98624us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762884.101441:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1100 00000100:00100000:1.0:1789762884.101448:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407401973851 time 407402022564 state 0x2 ref 0 00000100:00100000:0.0:1789762884.103344:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762884.103399:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015089152 00000100:00100000:3.0:1789762884.103409:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015089152 00000100:00100000:3.0:1789762884.103449:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407401973851 time 407402022564 state 0x0 ref 1 00000100:00100000:3.0:1789762884.103460:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407502022564 time 407402022564 state 0x1 ref 1 00000100:00100000:1.0:1789762884.198210:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407502022564 time 407502106633 state 0x0 ref 1 00000100:00100000:1.0:1789762884.198234:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe70e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015089152:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762884.198270:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0510d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.198289:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.198292:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.198297:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0510d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.198307:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0510d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.201146:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0510d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.201155:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0510d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.201160:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0510d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.201165:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762884.201168:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0510d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.201175:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0510d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.201188:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0510d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.201197:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0510d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762884.201242:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe70e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015089152:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3018us (97902us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762884.201249:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1101 00000100:00100000:1.0:1789762884.201256:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407502022564 time 407502106633 state 0x2 ref 0 00000100:00100000:2.0:1789762884.203343:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762884.203388:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015089280 00000100:00100000:3.0:1789762884.203397:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015089280 00000100:00100000:3.0:1789762884.203434:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407502022564 time 407502106633 state 0x0 ref 1 00000100:00100000:3.0:1789762884.203444:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407602106633 time 407502106633 state 0x1 ref 1 00000100:00100000:3.0:1789762884.298287:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407602106633 time 407602180931 state 0x0 ref 1 00000100:00100000:3.0:1789762884.298315:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015089280:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762884.298351:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05114 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.298365:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762884.298369:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762884.298373:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05114 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.298385:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05114 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.301015:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05114 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.301025:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05114 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.301033:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05114 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.301039:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762884.301043:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05114 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.301049:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05114 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.301063:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05114 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.301069:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05114 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762884.301144:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015089280:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2836us (97801us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762884.301155:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1102 00000100:00100000:3.0:1789762884.301166:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407602106633 time 407602180931 state 0x2 ref 0 00000100:00100000:0.0:1789762884.302975:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762884.303027:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015089408 00000100:00100000:1.0:1789762884.303036:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015089408 00000100:00100000:1.0:1789762884.303080:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407602106633 time 407602180931 state 0x0 ref 1 00000100:00100000:1.0:1789762884.303117:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407702180931 time 407602180931 state 0x1 ref 1 00000100:00100000:1.0:1789762884.398343:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407702180931 time 407702240138 state 0x0 ref 1 00000100:00100000:1.0:1789762884.398364:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe70a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015089408:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762884.398398:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0511b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.398412:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.398416:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.398420:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0511b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.398436:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0511b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.401488:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0511b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.401499:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0511b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.401505:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0511b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.401510:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762884.401513:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0511b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.401520:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0511b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.401532:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0511b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.401537:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b0511b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762884.401602:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe70a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015089408:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3244us (98627us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762884.401609:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1103 00000100:00100000:1.0:1789762884.401616:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407702180931 time 407702240138 state 0x2 ref 0 00000100:00100000:2.0:1789762884.403512:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762884.403553:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015089536 00000100:00100000:3.0:1789762884.403560:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015089536 00000100:00100000:3.0:1789762884.403598:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407702180931 time 407702240138 state 0x0 ref 1 00000100:00100000:3.0:1789762884.403614:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407802240138 time 407702240138 state 0x1 ref 1 00000100:00100000:1.0:1789762884.498484:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407802240138 time 407802380009 state 0x0 ref 1 00000100:00100000:1.0:1789762884.498512:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015089536:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762884.498549:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05122 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.498562:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.498566:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.498571:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05122 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.498582:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05122 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.501805:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05122 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.501815:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05122 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.501827:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05122 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.501832:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762884.501836:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05122 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.501842:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05122 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.501856:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05122 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.501862:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05122 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762884.501910:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015089536:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3411us (98399us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762884.501917:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1104 00000100:00100000:1.0:1789762884.501924:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407802240138 time 407802380009 state 0x2 ref 0 00000100:00100000:2.0:1789762884.504016:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762884.504059:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015089664 00000100:00100000:3.0:1789762884.504068:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015089664 00000100:00100000:3.0:1789762884.504121:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407802240138 time 407802380009 state 0x0 ref 1 00000100:00100000:3.0:1789762884.504132:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407902380009 time 407802380009 state 0x1 ref 1 00010000:00010000:0.0:1789762884.535200:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762884.598549:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407902380009 time 407902444694 state 0x0 ref 1 00000100:00100000:3.0:1789762884.598575:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d08700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015089664:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762884.598611:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05129 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.598625:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762884.598628:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762884.598633:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05129 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.598645:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05129 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.601824:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05129 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.601834:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05129 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.601841:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05129 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.601846:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762884.601849:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05129 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.601856:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05129 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.601869:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05129 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.601875:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05129 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762884.601920:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d08700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015089664:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3353us (97906us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762884.601926:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1105 00000100:00100000:3.0:1789762884.601934:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407902380009 time 407902444694 state 0x2 ref 0 00000100:00100000:2.0:1789762884.603899:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762884.603932:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015089792 00000100:00100000:3.0:1789762884.603941:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015089792 00000100:00100000:3.0:1789762884.603973:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 407902380009 time 407902444694 state 0x0 ref 1 00000100:00100000:3.0:1789762884.603983:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408002444694 time 407902444694 state 0x1 ref 1 00000100:00100000:1.0:1789762884.698701:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408002444694 time 408002576571 state 0x0 ref 1 00000100:00100000:1.0:1789762884.698738:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d09c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015089792:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762884.698776:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05130 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.698790:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762884.698794:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.698798:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05130 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.698810:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05130 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.702679:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05130 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.702713:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05130 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.702720:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05130 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.702725:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762884.702740:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05130 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.702747:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05130 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.702787:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05130 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.702799:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05130 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762884.702875:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d09c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015089792:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4160us (98977us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762884.702882:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1106 00000100:00100000:1.0:1789762884.702888:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408002444694 time 408002576571 state 0x2 ref 0 00000100:00100000:1.0:1789762884.706614:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762884.706658:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762884.706690:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762884.706750:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762884.706772:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015089920 00000100:00100000:3.0:1789762884.706786:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015089920 00000100:00100000:2.0:1789762884.706799:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015090048 00000100:00100000:2.0:1789762884.706826:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015090048 00000100:00100000:1.0:1789762884.706839:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762884.706840:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015090176 00000100:00100000:0.0:1789762884.706847:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015090176 00000100:00100000:3.0:1789762884.706854:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 332 00000100:00100000:3.0:1789762884.706867:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850da850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015089920:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762884.706901:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015090304 00000100:00100000:2.0:1789762884.706910:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015090304 00000100:00100000:0.0:1789762884.706932:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 159 00000100:00100000:1.0:1789762884.706975:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 318 00000100:00100000:0.0:1789762884.706976:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015090176:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762884.706982:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850da850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015089920:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 110us (364us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762884.707002:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 319 00000100:00100000:1.0:1789762884.707015:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2366:x1876702015090048:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762884.707018:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 332 00000100:00100000:2.0:1789762884.707043:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015090304:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762884.707072:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015090176:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 113us (379us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762884.707081:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 159 00000100:00100000:1.0:1789762884.707093:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2366:x1876702015090048:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 96us (435us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762884.707099:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015090432 00000100:00100000:3.0:1789762884.707105:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015090432 00000100:00100000:2.0:1789762884.707155:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015090304:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 84us (376us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762884.707170:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 318 00000100:00100000:2.0:1789762884.707191:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 319 00000100:00080000:0.0:1789762884.707195:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762884.707242:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408002444694 time 408002576571 state 0x0 ref 1 00000100:00100000:3.0:1789762884.707270:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408102576571 time 408002576571 state 0x1 ref 1 00000100:00100000:0.0:1789762884.727362:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789762884.727388:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfe70000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022182144:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762884.727466:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762884.727522:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022182144 00000100:00100000:2.0:1789762884.727529:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022182144 00000100:00100000:2.0:1789762884.727599:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 133 00000100:00100000:2.0:1789762884.727618:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022182144:12345-0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1789762884.727641:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00000024:2.0:1789762884.727669:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762884.727785:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022182144:12345-0@lo:13:osp-pre-0-0.0 Request processed in 175us (322us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762884.727792:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 133 00000100:00100000:0.0:1789762884.727933:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfe70000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022182144:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762884.798878:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408102576571 time 408102769133 state 0x0 ref 1 00000100:00100000:1.0:1789762884.798917:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015090432:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762884.798962:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05137 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.798983:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.798990:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.798999:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05137 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.799017:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05137 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.803524:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05137 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.803547:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05137 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.803567:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05137 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.803583:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762884.803592:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05137 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.803610:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05137 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.803643:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05137 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.803660:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05137 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762884.803786:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015090432:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4876us (96942us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762884.803806:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1107 00000100:00100000:1.0:1789762884.803826:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408102576571 time 408102769133 state 0x2 ref 0 00000100:00100000:1.0:1789762884.807568:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762884.807653:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015090560 00000100:00100000:3.0:1789762884.807680:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015090560 00000100:00100000:3.0:1789762884.807782:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408102576571 time 408102769133 state 0x0 ref 1 00000100:00100000:3.0:1789762884.807810:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408202769133 time 408102769133 state 0x1 ref 1 00000100:00100000:3.0:1789762884.898998:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408202769133 time 408202877398 state 0x0 ref 1 00000100:00100000:3.0:1789762884.899045:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015090560:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762884.899153:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0513e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.899176:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762884.899182:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762884.899204:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0513e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.899224:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0513e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.904520:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0513e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.904542:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0513e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.904555:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0513e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.904566:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762884.904573:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0513e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.904585:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0513e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.904609:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0513e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762884.904620:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0513e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762884.904710:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015090560:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5681us (97143us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762884.904722:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1108 00000100:00100000:3.0:1789762884.904758:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408202769133 time 408202877398 state 0x2 ref 0 00000100:00100000:1.0:1789762884.908455:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762884.908527:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015090688 00000100:00100000:3.0:1789762884.908540:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015090688 00000100:00100000:3.0:1789762884.908600:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408202769133 time 408202877398 state 0x0 ref 1 00000100:00100000:3.0:1789762884.908619:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408302877398 time 408202877398 state 0x1 ref 1 00000100:00100000:1.0:1789762884.999169:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408302877398 time 408303056010 state 0x0 ref 1 00000100:00100000:1.0:1789762884.999214:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015090688:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762884.999281:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05145 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.999312:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.999319:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762884.999327:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05145 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762884.999354:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05145 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.005168:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05145 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.005192:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05145 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.005212:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05145 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.005228:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762885.005237:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05145 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.005257:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05145 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.005290:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05145 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.005306:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b05145 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762885.005404:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015090688:12345-192.168.203.48@tcp:3:dd.0 Request processed in 6207us (96950us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762885.005417:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1109 00000100:00100000:1.0:1789762885.005431:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408302877398 time 408303056010 state 0x2 ref 0 00000100:00100000:1.0:1789762885.009300:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762885.009457:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015090816 00000100:00100000:3.0:1789762885.009471:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015090816 00000100:00080000:1.0:1789762885.009545:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762885.009561:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408302877398 time 408303056010 state 0x0 ref 1 00000100:00100000:3.0:1789762885.009582:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408403056010 time 408303056010 state 0x1 ref 1 00000100:00100000:1.0:1789762885.099351:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408403056010 time 408403232398 state 0x0 ref 1 00000100:00100000:1.0:1789762885.099394:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015090816:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762885.099457:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b0514c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.099480:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762885.099486:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762885.099495:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b0514c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.099513:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b0514c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.105203:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b0514c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.105262:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b0514c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.105283:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b0514c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.105301:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762885.105310:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b0514c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.105329:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b0514c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.105361:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b0514c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.105373:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ef125789/0x416193e012b0514c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762885.105474:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015090816:12345-192.168.203.48@tcp:3:dd.0 Request processed in 6092us (96177us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762885.105488:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1110 00000100:00100000:2.0:1789762885.105503:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408403056010 time 408403232398 state 0x2 ref 0 00000100:00100000:0.0:1789762885.109180:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762885.109251:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015090944 00000100:00100000:3.0:1789762885.109292:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015090944 00000100:00100000:3.0:1789762885.109356:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408403056010 time 408403232398 state 0x0 ref 1 00000100:00100000:3.0:1789762885.109384:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408503232398 time 408403232398 state 0x1 ref 1 00000100:00100000:3.0:1789762885.199405:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408503232398 time 408503295380 state 0x0 ref 1 00000100:00100000:3.0:1789762885.199433:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe70000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015090944:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762885.199471:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05153 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.199492:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762885.199496:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762885.199501:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05153 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.199512:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05153 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.203215:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05153 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.203232:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05153 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.203240:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05153 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.203245:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762885.203250:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05153 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.203256:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05153 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.203272:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05153 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.203283:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05153 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762885.203345:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe70000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015090944:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3919us (94168us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762885.203356:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1111 00000100:00100000:3.0:1789762885.203366:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408503232398 time 408503295380 state 0x2 ref 0 00000100:00100000:0.0:1789762885.205576:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762885.205615:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015091072 00000100:00100000:3.0:1789762885.205626:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015091072 00000100:00100000:3.0:1789762885.205676:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408503232398 time 408503295380 state 0x0 ref 1 00000100:00100000:3.0:1789762885.205688:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408603295380 time 408503295380 state 0x1 ref 1 00000100:00100000:0.0:1789762885.239212:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789762885.239226:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d09f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022182272:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.239230:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022182656:0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.239256:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022182528:0@lo:400:kworker.0 00000100:00100000:1.0:1789762885.239264:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022182784:0@lo:400:kworker.0 00000100:00100000:3.0:1789762885.239299:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762885.239308:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762885.239339:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762885.239351:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfe73480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022182400:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762885.239358:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d0ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022182912:0@lo:400:kworker.0 00000100:00100000:1.0:1789762885.239363:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022182656 00000100:00100000:1.0:1789762885.239372:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022182656 00000100:00100000:0.0:1789762885.239393:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762885.239394:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762885.239403:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762885.239439:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022182528 00000100:00100000:2.0:1789762885.239442:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022182784 00000100:00100000:1.0:1789762885.239444:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022182528 00000100:00100000:2.0:1789762885.239447:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022182784 00000100:00100000:3.0:1789762885.239461:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022182272 00000100:00100000:3.0:1789762885.239467:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022182272 00000100:00100000:0.0:1789762885.239471:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 320 00000100:00100000:1.0:1789762885.239481:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 321 00000100:00100000:0.0:1789762885.239490:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe70700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022182656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.239495:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:1.0:1789762885.239496:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022182528:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762885.239505:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 333 00000100:00100000:2.0:1789762885.239510:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022182784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762885.239516:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022182272:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762885.239585:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022182528:12345-0@lo:400:kworker.0 Request processed in 96us (248us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762885.239594:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe70700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022182656:12345-0@lo:400:kworker.0 Request processed in 108us (281us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762885.239594:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022182272:12345-0@lo:400:kworker.0 Request processed in 80us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762885.239596:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022182784:12345-0@lo:400:kworker.0 Request processed in 89us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762885.239598:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 321 00000100:00100000:2.0:1789762885.239615:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:3.0:1789762885.239619:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 333 00000100:00100000:0.0:1789762885.239624:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 320 00000100:00080000:1.0:1789762885.239625:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762885.239654:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022182400 00000100:00100000:2.0:1789762885.239658:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022182400 00000100:00100000:1.0:1789762885.239669:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022182656:0@lo:400:kworker.0 00000100:00100000:3.0:1789762885.239674:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d09f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022182272:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.239685:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022182912 00000100:00100000:0.0:1789762885.239690:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022182912 00000100:00080000:1.0:1789762885.239706:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762885.239710:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 134 00000100:00100000:2.0:1789762885.239721:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022182400:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762885.239741:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022182528:0@lo:400:kworker.0 00000100:00100000:0.0:1789762885.239749:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:0.0:1789762885.239760:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022182912:12345-0@lo:400:kworker.0 00002000:00000024:2.0:1789762885.239762:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762885.239784:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022182784:0@lo:400:kworker.0 00000100:00100000:2.0:1789762885.239812:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022182400:12345-0@lo:13:osp-pre-1-0.0 Request processed in 93us (423us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762885.239819:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 134 00000100:00100000:0.0:1789762885.239824:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022182912:12345-0@lo:400:kworker.0 Request processed in 64us (419us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762885.239832:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:3.0:1789762885.239851:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d0ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022182912:0@lo:400:kworker.0 00000100:00100000:1.0:1789762885.239855:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfe73480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022182400:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762885.299477:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408603295380 time 408603373098 state 0x0 ref 1 00000100:00100000:1.0:1789762885.299502:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe72300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015091072:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762885.299536:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0515a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.299551:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762885.299554:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762885.299559:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0515a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.299571:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0515a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.302503:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0515a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.302515:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0515a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.302522:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0515a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.302528:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc4dfe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762885.302532:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0515a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.302539:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0515a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.302554:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0515a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.302559:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0515a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762885.302602:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe72300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015091072:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3109us (97031us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762885.302610:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1112 00000100:00100000:1.0:1789762885.302617:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408603295380 time 408603373098 state 0x2 ref 0 00000100:00100000:1.0:1789762885.304472:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762885.304514:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015091200 00000100:00100000:3.0:1789762885.304523:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015091200 00000100:00100000:3.0:1789762885.304560:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408603295380 time 408603373098 state 0x0 ref 1 00000100:00100000:3.0:1789762885.304571:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408703373098 time 408603373098 state 0x1 ref 1 00000100:00100000:2.0:1789762885.399583:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408703373098 time 408703480365 state 0x0 ref 1 00000100:00100000:2.0:1789762885.399602:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015091200:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762885.399636:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002fb56763/0x416193e012b05161 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.399649:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762885.399652:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762885.399658:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002fb56763/0x416193e012b05161 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.399694:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002fb56763/0x416193e012b05161 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.402857:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002fb56763/0x416193e012b05161 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.402868:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002fb56763/0x416193e012b05161 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.402875:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002fb56763/0x416193e012b05161 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.402880:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87c99000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762885.402884:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002fb56763/0x416193e012b05161 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.402890:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002fb56763/0x416193e012b05161 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.402903:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002fb56763/0x416193e012b05161 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.402908:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002fb56763/0x416193e012b05161 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762885.402966:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015091200:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3368us (98495us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762885.402973:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1113 00000100:00100000:2.0:1789762885.402980:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408703373098 time 408703480365 state 0x2 ref 0 00000100:00100000:1.0:1789762885.406194:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762885.406238:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015091328 00000100:00100000:3.0:1789762885.406246:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015091328 00000100:00100000:3.0:1789762885.406287:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408703373098 time 408703480365 state 0x0 ref 1 00000100:00100000:3.0:1789762885.406298:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408803480365 time 408703480365 state 0x1 ref 1 00000100:00100000:3.0:1789762885.499642:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408803480365 time 408803538221 state 0x0 ref 1 00000100:00100000:3.0:1789762885.499665:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015091328:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762885.499701:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05168 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.499719:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762885.499736:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762885.499743:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05168 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.499758:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05168 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.502931:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05168 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.502942:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05168 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.502949:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05168 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.502955:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762885.502959:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05168 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.502965:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05168 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.502980:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05168 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.502990:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05168 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762885.503036:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015091328:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3379us (96844us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762885.503045:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1114 00000100:00100000:3.0:1789762885.503053:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408803480365 time 408803538221 state 0x2 ref 0 00000100:00100000:0.0:1789762885.505273:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762885.505308:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015091456 00000100:00100000:3.0:1789762885.505317:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015091456 00000100:00100000:3.0:1789762885.505353:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408803480365 time 408803538221 state 0x0 ref 1 00000100:00100000:3.0:1789762885.505368:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408903538221 time 408803538221 state 0x1 ref 1 00010000:00010000:2.0:1789762885.559221:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762885.599711:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408903538221 time 408903608246 state 0x0 ref 1 00000100:00100000:1.0:1789762885.599749:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015091456:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762885.599783:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0516f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.599797:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762885.599801:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762885.599805:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0516f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.599816:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0516f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.602677:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0516f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.602690:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0516f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.602699:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0516f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.602707:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc4dfe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762885.602711:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0516f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.602734:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0516f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.602752:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0516f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.602760:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b0516f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762885.602817:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015091456:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3079us (97545us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762885.602827:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1115 00000100:00100000:1.0:1789762885.602837:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408903538221 time 408903608246 state 0x2 ref 0 00000100:00100000:2.0:1789762885.604756:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762885.604797:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015091584 00000100:00100000:3.0:1789762885.604806:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015091584 00000100:00100000:3.0:1789762885.604844:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 408903538221 time 408903608246 state 0x0 ref 1 00000100:00100000:3.0:1789762885.604855:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409003608246 time 408903608246 state 0x1 ref 1 00000100:00100000:2.0:1789762885.699802:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409003608246 time 409003699825 state 0x0 ref 1 00000100:00100000:2.0:1789762885.699823:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015091584:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762885.699855:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05176 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.699868:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762885.699871:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762885.699876:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05176 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.699888:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05176 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.702936:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05176 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.702962:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05176 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.702969:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05176 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.702975:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762885.702979:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05176 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.702985:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05176 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.702998:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05176 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762885.703003:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05176 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762885.703051:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015091584:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3235us (98296us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762885.703058:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1116 00000100:00100000:2.0:1789762885.703066:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409003608246 time 409003699825 state 0x2 ref 0 00000100:00100000:2.0:1789762885.704990:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762885.705046:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015091712 00000100:00100000:3.0:1789762885.705056:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015091712 00000100:00100000:3.0:1789762885.705131:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409003608246 time 409003699825 state 0x0 ref 1 00000100:00100000:3.0:1789762885.705144:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409103699825 time 409003699825 state 0x1 ref 1 00000100:00100000:3.0:1789762885.799869:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409103699825 time 409103765060 state 0x0 ref 1 00000100:00100000:3.0:1789762885.799892:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015091712:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762885.799929:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0517d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.799942:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762885.799946:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762885.799951:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0517d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.799962:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0517d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.802688:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0517d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.802697:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0517d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.802703:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0517d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.802709:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762885.802712:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0517d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.802732:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0517d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.802746:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0517d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762885.802751:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b0517d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762885.802795:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015091712:12345-192.168.203.48@tcp:3:dd.0 Request processed in 2911us (97807us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762885.802801:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1117 00000100:00100000:3.0:1789762885.802808:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409103699825 time 409103765060 state 0x2 ref 0 00000100:00100000:0.0:1789762885.804845:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762885.804891:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015091840 00000100:00100000:3.0:1789762885.804901:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015091840 00000100:00100000:3.0:1789762885.804943:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409103699825 time 409103765060 state 0x0 ref 1 00000100:00100000:3.0:1789762885.804954:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409203765060 time 409103765060 state 0x1 ref 1 00000100:00100000:1.0:1789762885.899949:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409203765060 time 409203844405 state 0x0 ref 1 00000100:00100000:1.0:1789762885.899971:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015091840:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762885.900007:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b05184 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.900020:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762885.900023:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762885.900028:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b05184 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.900044:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b05184 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.903275:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b05184 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.903287:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b05184 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.903295:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b05184 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.903301:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebc4dfe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762885.903304:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b05184 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.903311:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b05184 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.903326:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b05184 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762885.903332:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b8be6523/0x416193e012b05184 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762885.903384:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015091840:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3420us (98541us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762885.903391:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1118 00000100:00100000:1.0:1789762885.903398:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409203765060 time 409203844405 state 0x2 ref 0 00000100:00100000:1.0:1789762885.905557:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762885.905596:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015091968 00000100:00100000:3.0:1789762885.905605:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015091968 00000100:00100000:3.0:1789762885.905645:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409203765060 time 409203844405 state 0x0 ref 1 00000100:00100000:3.0:1789762885.905657:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409303844405 time 409203844405 state 0x1 ref 1 00000100:00100000:2.0:1789762886.000032:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409303844405 time 409303928116 state 0x0 ref 1 00000100:00100000:2.0:1789762886.000056:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015091968:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762886.000119:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b0518b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.000135:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762886.000140:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762886.000148:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b0518b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.000163:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b0518b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.003514:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b0518b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.003525:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b0518b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.003531:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b0518b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.003536:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aaa00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762886.003539:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b0518b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.003556:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b0518b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.003569:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b0518b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.003574:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b0518b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762886.003620:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015091968:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3573us (98064us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762886.003627:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1119 00000100:00100000:2.0:1789762886.003634:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409303844405 time 409303928116 state 0x2 ref 0 00000100:00100000:2.0:1789762886.005795:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762886.005845:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015092096 00000100:00100000:3.0:1789762886.005855:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015092096 00000100:00100000:3.0:1789762886.005895:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409303844405 time 409303928116 state 0x0 ref 1 00000100:00100000:3.0:1789762886.005906:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409403928116 time 409303928116 state 0x1 ref 1 00000100:00100000:3.0:1789762886.100117:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409403928116 time 409404012760 state 0x0 ref 1 00000100:00100000:3.0:1789762886.100139:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015092096:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762886.100175:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05192 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.100188:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762886.100192:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762886.100199:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05192 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.100214:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05192 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.103666:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05192 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.103680:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05192 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.103694:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05192 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.103702:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762886.103707:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05192 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.103733:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05192 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.103752:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05192 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.103760:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b05192 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762886.103806:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015092096:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3674us (98013us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762886.103813:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1120 00000100:00100000:3.0:1789762886.103820:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409403928116 time 409404012760 state 0x2 ref 0 00000100:00100000:2.0:1789762886.106121:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762886.106153:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015092224 00000100:00100000:3.0:1789762886.106161:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015092224 00000100:00100000:3.0:1789762886.106197:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409403928116 time 409404012760 state 0x0 ref 1 00000100:00100000:3.0:1789762886.106211:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409504012760 time 409404012760 state 0x1 ref 1 00000100:00100000:1.0:1789762886.200229:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409504012760 time 409504125845 state 0x0 ref 1 00000100:00100000:1.0:1789762886.200251:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc3c3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015092224:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762886.200287:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2bf38ef/0x416193e012b05199 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.200300:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762886.200304:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762886.200309:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2bf38ef/0x416193e012b05199 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.200319:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2bf38ef/0x416193e012b05199 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.203376:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2bf38ef/0x416193e012b05199 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.203387:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2bf38ef/0x416193e012b05199 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.203395:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2bf38ef/0x416193e012b05199 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.203400:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e98e02c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762886.203404:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2bf38ef/0x416193e012b05199 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.203410:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2bf38ef/0x416193e012b05199 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.203423:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2bf38ef/0x416193e012b05199 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.203428:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2bf38ef/0x416193e012b05199 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762886.203476:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc3c3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015092224:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3233us (97356us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762886.203483:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1121 00000100:00100000:1.0:1789762886.203490:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409504012760 time 409504125845 state 0x2 ref 0 00000100:00100000:1.0:1789762886.205460:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762886.205494:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015092352 00000100:00100000:3.0:1789762886.205502:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015092352 00000100:00100000:3.0:1789762886.205538:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409504012760 time 409504125845 state 0x0 ref 1 00000100:00100000:3.0:1789762886.205549:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409604125845 time 409504125845 state 0x1 ref 1 00000100:00100000:2.0:1789762886.300335:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409604125845 time 409604225918 state 0x0 ref 1 00000100:00100000:2.0:1789762886.300364:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015092352:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762886.300405:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b051a0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.300424:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762886.300428:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762886.300447:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b051a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.300463:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b051a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.303633:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b051a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.303644:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b051a0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.303652:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b051a0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.303657:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aaa00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762886.303661:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b051a0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.303667:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b051a0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.303682:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b051a0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.303687:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b051a0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762886.303761:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015092352:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3408us (98300us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762886.303769:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1122 00000100:00100000:2.0:1789762886.303776:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409604125845 time 409604225918 state 0x2 ref 0 00000100:00100000:0.0:1789762886.305755:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762886.305804:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015092480 00000100:00100000:3.0:1789762886.305812:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015092480 00000100:00100000:3.0:1789762886.305850:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409604125845 time 409604225918 state 0x0 ref 1 00000100:00100000:3.0:1789762886.305861:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409704225918 time 409604225918 state 0x1 ref 1 00000100:00100000:3.0:1789762886.400393:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409704225918 time 409704289072 state 0x0 ref 1 00000100:00100000:3.0:1789762886.400414:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe70380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015092480:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:3.0:1789762886.400448:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b051a7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.400462:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762886.400466:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762886.400471:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b051a7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.400486:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b051a7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.403640:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b051a7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.403651:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b051a7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.403657:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b051a7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.403663:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762886.403666:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b051a7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.403672:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b051a7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.403685:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b051a7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762886.403690:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2cc60cf/0x416193e012b051a7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762886.403760:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe70380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015092480:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3349us (98005us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762886.403771:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1123 00000100:00100000:3.0:1789762886.403780:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409704225918 time 409704289072 state 0x2 ref 0 00000100:00100000:0.0:1789762886.405673:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762886.405711:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015092608 00000100:00100000:3.0:1789762886.405735:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015092608 00000100:00100000:3.0:1789762886.405779:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409704225918 time 409704289072 state 0x0 ref 1 00000100:00100000:3.0:1789762886.405792:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409804289072 time 409704289072 state 0x1 ref 1 00000100:00100000:1.0:1789762886.500505:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409804289072 time 409804398225 state 0x0 ref 1 00000100:00100000:1.0:1789762886.500540:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe73b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015092608:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:1.0:1789762886.500581:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b67ee2f4/0x416193e012b051ae lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.500599:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762886.500604:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762886.500611:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b67ee2f4/0x416193e012b051ae lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.500626:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b67ee2f4/0x416193e012b051ae lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.503698:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b67ee2f4/0x416193e012b051ae lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.503711:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b67ee2f4/0x416193e012b051ae lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.503735:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b67ee2f4/0x416193e012b051ae lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.503744:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762886.503750:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b67ee2f4/0x416193e012b051ae lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.503777:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b67ee2f4/0x416193e012b051ae lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.503797:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b67ee2f4/0x416193e012b051ae lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.503807:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b67ee2f4/0x416193e012b051ae lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762886.503863:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe73b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015092608:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3334us (98189us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762886.503872:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1124 00000100:00100000:1.0:1789762886.503882:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409804289072 time 409804398225 state 0x2 ref 0 00000100:00100000:1.0:1789762886.505977:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762886.506028:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015092736 00000100:00100000:3.0:1789762886.506036:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015092736 00000100:00100000:3.0:1789762886.506080:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409804289072 time 409804398225 state 0x0 ref 1 00000100:00100000:3.0:1789762886.506118:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409904398225 time 409804398225 state 0x1 ref 1 00010000:00010000:3.0:1789762886.583217:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762886.600571:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409904398225 time 409904467558 state 0x0 ref 1 00000100:00100000:2.0:1789762886.600596:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015092736:12345-192.168.203.48@tcp:3:dd.0 00010000:00010000:2.0:1789762886.600634:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b051b5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.600648:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762886.600652:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762886.600656:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b051b5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.600667:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b051b5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.603724:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b051b5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.603741:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b051b5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.603781:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b051b5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.603787:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762886.603791:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b051b5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.603797:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b051b5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.603810:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b051b5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.603816:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b051b5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762886.603863:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015092736:12345-192.168.203.48@tcp:3:dd.0 Request processed in 3277us (97889us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762886.603869:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1125 00000100:00100000:2.0:1789762886.603877:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409904398225 time 409904467558 state 0x2 ref 0 00000100:00100000:2.0:1789762886.644974:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762886.645054:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015092864 00000100:00100000:1.0:1789762886.645067:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015092864 00000100:00100000:1.0:1789762886.645159:0:6664:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 29 00000100:00100000:1.0:1789762886.645171:0:6664:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d08e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:6050:x1876702015092864:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762886.645188:0:6664:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762886.645226:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b04ef9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd719 expref: 6 pid: 6649 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.645261:0:6664:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762886.645314:0:6664:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d08e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:6050:x1876702015092864:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 146us (347us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762886.645324:0:6664:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 29 00000100:00100000:0.0:1789762886.658133:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762886.658189:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015092992 00000100:00100000:1.0:1789762886.658195:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015092992 00000100:00100000:1.0:1789762886.658252:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 321 00000100:00080000:2.0:1789762886.658252:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762886.658268:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe72d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:12244:x1876702015092992:12345-192.168.203.48@tcp:101:bash.0 00010000:00010000:1.0:1789762886.658283:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762886.658301:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b051bc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd720 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.658358:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b051c3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.658372:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b051c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.658384:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b051c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762886.658418:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00000004:00200000:1.0:1789762886.658423:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762886.658440:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b051c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.658474:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b051bc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd720 expref: 11 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.658491:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b051c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd720 expref: 10 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.658507:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:1.0:1789762886.658510:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762886.658513:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4d800, rc 0) 00000100:00100000:1.0:1789762886.658550:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe72d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:12244:x1876702015092992:12345-192.168.203.48@tcp:101:bash.0 Request processed in 285us (418us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762886.658558:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 321 00000100:00100000:0.0:1789762886.664491:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762886.664542:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015093248 00000100:00100000:2.0:1789762886.664548:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015093248 00000100:00100000:2.0:1789762886.664590:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 322 00000100:00100000:2.0:1789762886.664598:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe73800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:14861:x1876702015093248:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:2.0:1789762886.664606:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762886.664615:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b051ca lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd727 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762886.664658:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00010000:00010000:2.0:1789762886.664798:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b051ca lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd727 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.664807:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c500) 00010000:00010000:2.0:1789762886.664809:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762886.664812:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e81482c00/0x416193e012b051ca lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd727 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.664821:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81482c00, rc 0) 00000100:00100000:2.0:1789762886.664848:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe73800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14861:x1876702015093248:12345-192.168.203.48@tcp:101:rm.0 Request processed in 253us (359us total) trans 4294967571 rc 301/301 00000100:00100000:2.0:1789762886.664854:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 322 00000100:00100000:2.0:1789762886.667009:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762886.667049:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015093376 00000100:00100000:3.0:1789762886.667053:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015093376 00000100:00100000:3.0:1789762886.667082:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 101 00000100:00100000:3.0:1789762886.667114:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14861:x1876702015093376:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:3.0:1789762886.669364:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14861:x1876702015093376:12345-192.168.203.48@tcp:37:rm.0 Request processed in 2273us (2351us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762886.669377:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 101 00000100:00100000:0.0:1789762886.671313:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762886.671356:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015093632 00000100:00100000:2.0:1789762886.671362:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015093632 00000100:00100000:2.0:1789762886.671389:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 102 00000100:00100000:2.0:1789762886.671396:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe72a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:14861:x1876702015093632:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:2.0:1789762886.671489:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe72a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14861:x1876702015093632:12345-192.168.203.48@tcp:35:rm.0 Request processed in 93us (177us total) trans 4294967572 rc 0/0 00000100:00100000:2.0:1789762886.671495:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 102 00000100:00100000:2.0:1789762886.672624:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762886.672683:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015093888 00000100:00100000:1.0:1789762886.672690:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015093888 00000100:00100000:1.0:1789762886.672749:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 323 00000100:00100000:1.0:1789762886.672761:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d08000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:14861:x1876702015093888:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:1.0:1789762886.672773:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762886.672784:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b051d8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd72e expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.672829:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b051df lrc: 3/1,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.672840:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b051df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.672850:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b051df lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762886.672859:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00010000:00010000:1.0:1789762886.672918:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b051d8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd72e expref: 15 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.672938:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b051df lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd72e expref: 14 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.672947:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:1.0:1789762886.672950:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1789762886.672952:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4d000, rc 0) 00000100:00100000:1.0:1789762886.672995:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d08000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14861:x1876702015093888:12345-192.168.203.48@tcp:101:rm.0 Request processed in 237us (371us total) trans 4294967573 rc 0/0 00000100:00100000:1.0:1789762886.673003:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 323 00000100:00100000:0.0:1789762886.674282:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762886.674316:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015094016 00000100:00100000:1.0:1789762886.674323:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015094016 00000100:00100000:1.0:1789762886.674359:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 324 00000100:00100000:1.0:1789762886.674369:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe72680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14861:x1876702015094016:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:1.0:1789762886.674399:0:5806:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7610564608 00000100:00100000:1.0:1789762886.674435:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe72680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14861:x1876702015094016:12345-192.168.203.48@tcp:41:rm.0 Request processed in 67us (152us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762886.674443:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 324 00000100:00100000:0.0:1789762886.675570:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762886.675604:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015094272 00000100:00100000:3.0:1789762886.675609:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015094272 00000100:00100000:3.0:1789762886.675645:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 103 00000100:00100000:3.0:1789762886.675663:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14861:x1876702015094272:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762886.675851:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:14861:x1876702015094272:12345-192.168.203.48@tcp:35:rm.0 Request processed in 192us (281us total) trans 4294967574 rc 0/0 00000100:00100000:3.0:1789762886.675857:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 103 00000100:00100000:2.0:1789762886.676880:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762886.676925:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015094528 00000100:00100000:0.0:1789762886.676931:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015094528 00000100:00100000:0.0:1789762886.676969:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 325 00000100:00100000:0.0:1789762886.676980:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:14861:x1876702015094528:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:0.0:1789762886.677021:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1789762886.677037:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b04eeb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd712 expref: 15 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.677058:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfeb5000/0x416193e012b04c1a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd70b expref: 14 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.677068:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff981ec2179400) 00010000:00010000:0.0:1789762886.677082:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b051c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd720 expref: 13 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.677136:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:0.0:1789762886.677152:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762886.677165:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b051ed lrc: 3/0,1 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.677172:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b051ed lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.677178:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b051ed lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.677188:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b051f4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.677193:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b051f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.677197:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b051f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.677573:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b051fb lrc: 3/0,1 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.677583:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b051fb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.677591:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b051fb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762886.678773:0:5805:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].21, 64 off9536 00000004:00080000:0.0:1789762886.678781:0:5805:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 409982693589ns, next at 410982692121ns, rc = 0 00010000:00010000:0.0:1789762886.678835:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05202 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.678846:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05202 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.678855:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05202 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.678863:0:5805:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05202 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.678871:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05202 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.678879:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05202 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.678899:0:5805:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762886.678912:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b051fb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.678918:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762886.678919:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b051fb lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.678929:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b051fb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.678930:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05202 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.678937:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762886.678942:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b051fb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.678946:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762886.678949:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b051fb lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.678953:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05202 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.678977:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05202 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.678980:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff981ea0aa3c00) 00010000:00010000:0.0:1789762886.678987:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762886.678991:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b051fb lrc: 2/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.678995:0:5793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff981ea0aa3c00) 00010000:00010000:0.0:1789762886.678997:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b051fb lrc: 0/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.679000:0:5793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789762886.679006:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05202 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x44819480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.679011:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05202 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.679015:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b051ed lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.679022:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b051ed lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.679024:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762886.679027:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762886.679029:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b051ed lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.679030:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762886.679037:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762886.679040:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b051ed lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.679047:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b051ed lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.679056:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:0.0:1789762886.679059:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762886.679062:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b051ed lrc: 2/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762886.679069:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b051ed lrc: 0/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762886.679114:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001d7388ae reply state 0000000014dabecd transno 4294967575 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b051f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762886.679123:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762886.679180:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14861:x1876702015094528:12345-192.168.203.48@tcp:36:rm.0 Request processed in 2202us (2300us total) trans 4294967575 rc 0/0 00000100:00100000:0.0:1789762886.679190:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 325 00010000:00010000:1.0:1789762886.679982:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b051f4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.679988:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b051f4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.679993:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b051f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.679996:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762886.679999:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b051f4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.680003:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b051f4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.680009:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x4009f790 (ffff981ea0aa3d00) 00010000:00010000:1.0:1789762886.680011:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762886.680013:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b051f4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.680018:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b051f4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762886.680569:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762886.680612:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015094784 00000100:00100000:0.0:1789762886.680617:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015094784 00000100:00100000:0.0:1789762886.680648:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 104 00000100:00100000:0.0:1789762886.680654:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:14861:x1876702015094784:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:0.0:1789762886.680753:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14861:x1876702015094784:12345-192.168.203.48@tcp:35:rm.0 Request processed in 99us (184us total) trans 4294967576 rc 0/0 00000100:00100000:0.0:1789762886.680759:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 104 00000100:00100000:0.0:1789762886.681607:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762886.681640:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015095040 00000100:00100000:1.0:1789762886.681646:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015095040 00000100:00100000:1.0:1789762886.681680:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 326 00000100:00100000:1.0:1789762886.681687:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe73100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:14861:x1876702015095040:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:1.0:1789762886.681708:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1789762886.681733:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b051df lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd72e expref: 14 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.681757:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3800/0x416193e012b04bfe lrc: 0/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd704 expref: 13 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.681765:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:1.0:1789762886.681773:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b04b9c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd6ef expref: 12 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.681779:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c500) 00010000:00010000:1.0:1789762886.681782:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762886.681792:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05209 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.681798:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05209 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.681803:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05209 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.681814:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05210 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.681818:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05210 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.681822:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05210 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682038:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05217 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682044:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05217 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682049:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05217 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762886.682411:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00010000:00010000:1.0:1789762886.682418:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05217 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682423:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05217 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682427:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05217 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682431:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4da00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762886.682433:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05217 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682440:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05217 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682447:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff981ec2179100) 00010000:00010000:1.0:1789762886.682448:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762886.682450:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05217 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682454:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05217 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682465:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05209 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682469:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05209 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682473:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05209 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682476:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4c400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762886.682478:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05209 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682482:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05209 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682486:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:1.0:1789762886.682487:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762886.682489:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05209 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762886.682496:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05209 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762886.682502:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000007f70fd30 reply state 000000004536391e transno 4294967577 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05210 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762886.682506:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762886.682552:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe73100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:14861:x1876702015095040:12345-192.168.203.48@tcp:36:rm.0 Request processed in 865us (953us total) trans 4294967577 rc 0/0 00000100:00100000:1.0:1789762886.682558:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 326 00010000:00010000:2.0:1789762886.683192:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05210 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.683203:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05210 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.683211:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05210 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.683217:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4cc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762886.683221:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05210 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.683229:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05210 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.683240:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179700) 00010000:00010000:2.0:1789762886.683243:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762886.683246:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05210 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762886.683251:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05210 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762887.608176:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1789762887.634674:0:16464:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 410938583311ns, next at 410982692121ns, rc = 0 00000001:00080000:2.0:1789762887.644671:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967576 is committed 00000004:00080000:2.0:1789762887.644689:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 27 committed 00000001:00080000:2.0:1789762887.644715:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967577 is committed 00000004:00080000:2.0:1789762887.644729:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 28 committed 00000100:00100000:0.0:1789762887.644831:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789762887.644852:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022183040:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789762887.644911:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762887.644949:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022183040 00000100:00100000:1.0:1789762887.644956:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022183040 00000100:00080000:2.0:1789762887.645002:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762887.645027:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:1.0:1789762887.645041:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb494fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022183040:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1789762887.645068:0:6648:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:21 count 1 00002000:00010000:1.0:1789762887.645119:0:6648:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 104857600, mtime: 0x6aad9d3c, atime: 0x0, ctime: 0x6aad9d3c, blocks: 0x3200b 00010000:00010000:1.0:1789762887.645127:0:6648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b0521e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762887.645141:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=410s v=(0 0) 00010000:00010000:1.0:1789762887.645145:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762887.645149:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b0521e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762887.645163:0:6648:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b0521e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762887.645174:0:6648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b0521e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762887.645180:0:6648:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b0521e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762887.645194:0:6648:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762887.645218:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762887.645225:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b0521e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762887.645233:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762887.645237:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b0521e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762887.645244:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b0521e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762887.645258:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b0521e lrc: 1/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762887.645262:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000043c9dff6/0x416193e012b0521e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762887.645275:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762887.645276:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762887.645277:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762887.645569:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb494fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022183040:12345-0@lo:6:osp-syn-0-0.0 Request processed in 532us (656us total) trans 4294968132 rc 0/0 00000100:00100000:1.0:1789762887.645577:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000004:00080000:0.0:1789762887.645612:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebfb1ce00/2, rc 0, transno 4294968132 00000100:00100000:0.0:1789762887.645625:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022183040:0@lo:6:osp-syn-0-0.0 00000004:00080000:2.0:1789762887.645798:0:16464:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 410949710554ns, next at 358343111120ns, rc = 0 00000004:00080000:1.0:1789762887.646685:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 22 committed 00010000:00010000:3.0:1789762888.631400:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762889.655256:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762889.826512:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762889.826552:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762889.826586:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762889.826626:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762889.826656:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015095296 00000100:00100000:2.0:1789762889.826666:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015095296 00000100:00100000:3.0:1789762889.826667:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015095168 00000100:00100000:0.0:1789762889.826670:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015095552 00000100:00100000:3.0:1789762889.826676:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015095168 00000100:00100000:1.0:1789762889.826683:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015095424 00000100:00100000:0.0:1789762889.826685:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015095552 00000100:00100000:1.0:1789762889.826715:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015095424 00000100:00100000:2.0:1789762889.826759:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 323 00000100:00100000:3.0:1789762889.826766:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 334 00000100:00100000:0.0:1789762889.826775:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 324 00000100:00100000:2.0:1789762889.826783:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe70a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015095296:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762889.826787:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015095168:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762889.826790:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe70000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015095552:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762889.826809:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 161 00000100:00100000:1.0:1789762889.826822:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe70e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015095424:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762889.826862:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe70000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2364:x1876702015095552:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 77us (233us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762889.826865:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe70a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015095296:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 92us (276us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762889.826872:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015095168:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 91us (365us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762889.826875:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 324 00000100:00100000:3.0:1789762889.826887:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 334 00000100:00100000:2.0:1789762889.826902:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 323 00000100:00100000:1.0:1789762889.826911:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe70e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015095424:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 86us (351us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762889.826928:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 161 00000100:00080000:2.0:1789762889.826930:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789762889.826937:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762889.847239:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfe72300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022183168:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762889.847288:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762889.847327:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022183168 00000100:00100000:3.0:1789762889.847334:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022183168 00000100:00080000:0.0:1789762889.847386:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762889.847390:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 135 00000100:00100000:3.0:1789762889.847398:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022183168:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762889.847424:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3754950656 avail 3752853504 00000100:00100000:3.0:1789762889.847463:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022183168:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (176us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762889.847470:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 135 00000100:00100000:2.0:1789762889.847504:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfe72300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022183168:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762890.359312:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022183552:0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.359317:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022183680:0@lo:400:kworker.0 00000100:00100000:3.0:1789762890.359339:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfe71f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022183424:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762890.359394:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762890.359400:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762890.359413:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022183296:0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.359458:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022183552 00000100:00100000:1.0:1789762890.359461:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762890.359470:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022183552 00000100:00100000:3.0:1789762890.359484:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022183936:0@lo:400:kworker.0 00000100:00100000:2.0:1789762890.359506:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022183424 00000100:00100000:2.0:1789762890.359511:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022183424 00000100:00100000:1.0:1789762890.359517:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022183808:0@lo:400:kworker.0 00000100:00100000:3.0:1789762890.359528:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762890.359547:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762890.359552:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 325 00000100:00100000:0.0:1789762890.359568:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022183552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762890.359575:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 136 00000100:00100000:2.0:1789762890.359586:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022183424:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762890.359597:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762890.359602:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022183296 00000100:00100000:1.0:1789762890.359606:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022183296 00002000:00000024:2.0:1789762890.359618:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762890.359644:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 335 00000100:00100000:3.0:1789762890.359645:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022183808 00000100:00100000:3.0:1789762890.359651:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022183808 00000100:00100000:1.0:1789762890.359659:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850db850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022183296:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.359660:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022183552:12345-0@lo:400:kworker.0 Request processed in 92us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762890.359670:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022183424:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762890.359677:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 325 00000100:00100000:2.0:1789762890.359682:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 136 00000100:00100000:3.0:1789762890.359682:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022183680 00000100:00100000:3.0:1789762890.359701:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022183680 00000100:00100000:0.0:1789762890.359727:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:1.0:1789762890.359733:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850db850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022183296:12345-0@lo:400:kworker.0 Request processed in 79us (272us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762890.359738:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 327 00000100:00100000:0.0:1789762890.359745:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022183808:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762890.359751:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 335 00000100:00100000:2.0:1789762890.359755:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022183552:0@lo:400:kworker.0 00000100:00100000:3.0:1789762890.359758:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022183680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.359805:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022183808:12345-0@lo:400:kworker.0 Request processed in 66us (257us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762890.359815:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:1.0:1789762890.359822:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022183296:0@lo:400:kworker.0 00000100:00100000:3.0:1789762890.359829:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022183680:12345-0@lo:400:kworker.0 Request processed in 76us (230us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762890.359837:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 327 00000100:00100000:2.0:1789762890.359848:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfe71f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022183424:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762890.359849:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022183808:0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.359853:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022183936 00000100:00100000:0.0:1789762890.359862:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022183936 00000100:00100000:3.0:1789762890.359882:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022183680:0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.359910:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00080000:3.0:1789762890.359916:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789762890.359922:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762890.359923:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022183936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762890.359972:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022183936:12345-0@lo:400:kworker.0 Request processed in 53us (444us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762890.359980:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00100000:2.0:1789762890.359998:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022183936:0@lo:400:kworker.0 00010000:00010000:3.0:1789762890.679263:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762891.703285:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789762892.426030:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968132 is committed 00010000:00010000:3.0:1789762892.727281:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762893.752179:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762894.775290:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762894.946178:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762894.946223:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762894.946256:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015095680 00000100:00100000:3.0:1789762894.946265:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762894.946276:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015095680 00000100:00100000:3.0:1789762894.946338:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762894.946364:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 336 00000100:00100000:2.0:1789762894.946390:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015095680:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762894.946415:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015095936 00000100:00100000:3.0:1789762894.946427:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015095936 00000100:00100000:0.0:1789762894.946472:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015095808 00000100:00100000:0.0:1789762894.946480:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015095808 00000100:00100000:3.0:1789762894.946507:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 163 00000100:00100000:2.0:1789762894.946509:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015095680:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 115us (324us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762894.946529:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015096064 00000100:00100000:2.0:1789762894.946531:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 336 00000100:00100000:3.0:1789762894.946538:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015095936:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762894.946539:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015096064 00000100:00100000:0.0:1789762894.946597:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 328 00000100:00100000:0.0:1789762894.946608:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015095808:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762894.946618:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015095936:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 100us (395us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762894.946623:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 163 00000100:00080000:3.0:1789762894.946676:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762894.946682:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015095808:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 73us (418us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762894.946684:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762894.946697:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 328 00000100:00100000:3.0:1789762894.946723:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 329 00000100:00100000:3.0:1789762894.946740:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015096064:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762894.946780:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015096064:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 46us (441us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762894.946789:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 329 00000100:00100000:3.0:1789762894.967244:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789762894.967264:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfe71880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022184064:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762894.967310:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762894.967372:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022184064 00000100:00100000:2.0:1789762894.967382:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022184064 00000100:00080000:3.0:1789762894.967455:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762894.967501:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 137 00000100:00100000:2.0:1789762894.967519:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb59450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022184064:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762894.967567:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762894.967705:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb59450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022184064:12345-0@lo:13:osp-pre-0-0.0 Request processed in 174us (374us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762894.967711:0:3271:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294968132 gen 2 00000100:00100000:2.0:1789762894.967717:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000004:00080000:3.0:1789762894.967718:0:3271:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebfb1ce00, transno 4294968132 00000100:00100000:3.0:1789762894.967749:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfe71880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022184064:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762895.479355:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfe73480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022184192:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762895.479356:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022184320:0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.479380:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d09c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022184448:0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.479455:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022184576:0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.479514:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762895.479536:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762895.479547:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762895.479618:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022184448 00000100:00100000:3.0:1789762895.479631:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022184448 00000100:00100000:0.0:1789762895.479741:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022184704:0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.479754:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:1.0:1789762895.479765:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d0b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022184832:0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.479766:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762895.479777:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022184448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.479850:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762895.479874:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762895.479874:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022184576 00000100:00100000:2.0:1789762895.479888:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022184576 00000100:00100000:3.0:1789762895.479909:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022184448:12345-0@lo:400:kworker.0 Request processed in 132us (382us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762895.479932:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00100000:2.0:1789762895.479975:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022184704 00000100:00100000:2.0:1789762895.479980:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022184704 00000100:00100000:3.0:1789762895.479989:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 331 00000100:00100000:1.0:1789762895.479990:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022184320 00000100:00100000:1.0:1789762895.480001:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022184320 00000100:00100000:3.0:1789762895.480003:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022184576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.480046:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:2.0:1789762895.480065:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022184704:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762895.480094:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:3.0:1789762895.480122:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022184576:12345-0@lo:400:kworker.0 Request processed in 85us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762895.480134:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 331 00000100:00100000:1.0:1789762895.480149:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022184320:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762895.480171:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022184832 00000100:00100000:2.0:1789762895.480177:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022184704:12345-0@lo:400:kworker.0 Request processed in 115us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762895.480182:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022184832 00000100:00080000:3.0:1789762895.480188:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762895.480199:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:3.0:1789762895.480211:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022184192 00000100:00100000:3.0:1789762895.480219:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022184192 00000100:00100000:1.0:1789762895.480240:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022184320:12345-0@lo:400:kworker.0 Request processed in 132us (717us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762895.480257:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00100000:0.0:1789762895.480298:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:2.0:1789762895.480309:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022184576:0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.480315:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:0.0:1789762895.480321:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022184832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762895.480351:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022184192:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762895.480375:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022184320:0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.480383:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d09c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022184448:0@lo:400:kworker.0 00002000:00000024:3.0:1789762895.480403:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762895.480418:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022184832:12345-0@lo:400:kworker.0 Request processed in 103us (544us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762895.480433:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00100000:1.0:1789762895.480443:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d0b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022184832:0@lo:400:kworker.0 00000100:00080000:2.0:1789762895.480467:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762895.480487:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022184192:12345-0@lo:13:osp-pre-1-0.0 Request processed in 146us (976us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762895.480496:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:2.0:1789762895.480519:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022184704:0@lo:400:kworker.0 00000100:00100000:2.0:1789762895.480549:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfe73480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022184192:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789762895.799321:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762896.823343:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762897.847212:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762898.871265:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762899.895307:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762900.066445:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762900.066490:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762900.066520:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015096192 00000100:00100000:3.0:1789762900.066522:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762900.066535:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015096192 00000100:00100000:0.0:1789762900.066545:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015096320 00000100:00100000:0.0:1789762900.066552:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015096320 00000100:00100000:3.0:1789762900.066579:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762900.066596:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015096576 00000100:00100000:2.0:1789762900.066604:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 338 00000100:00100000:1.0:1789762900.066605:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015096576 00000100:00100000:0.0:1789762900.066628:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 333 00000100:00100000:2.0:1789762900.066631:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015096192:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762900.066643:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015096320:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:3.0:1789762900.066668:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762900.066680:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 334 00000100:00100000:1.0:1789762900.066691:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015096576:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762900.066711:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015096448 00000100:00100000:3.0:1789762900.066717:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015096448 00000100:00100000:2.0:1789762900.066725:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015096192:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 110us (293us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762900.066726:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015096320:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 86us (234us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762900.066733:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015096576:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 42us (209us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762900.066738:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 338 00000100:00100000:1.0:1789762900.066749:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 334 00000100:00100000:0.0:1789762900.066764:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 333 00000100:00100000:3.0:1789762900.066771:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 165 00000100:00100000:3.0:1789762900.066787:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015096448:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:0.0:1789762900.066807:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762900.066831:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015096448:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 49us (253us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762900.066840:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 165 00000100:00100000:3.0:1789762900.087273:0:3272:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789762900.087294:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfe71880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022184960:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762900.087340:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762900.087377:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022184960 00000100:00100000:2.0:1789762900.087384:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022184960 00000100:00080000:0.0:1789762900.087412:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762900.087441:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 139 00000100:00100000:2.0:1789762900.087454:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022184960:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762900.087487:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762900.087530:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022184960:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (188us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762900.087537:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 139 00000100:00100000:3.0:1789762900.087570:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfe71880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022184960:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762900.599328:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d09c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022185344:0@lo:400:kworker.0 00000100:00100000:2.0:1789762900.599334:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfe71f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022185216:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762900.599347:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022185472:0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.599359:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022185088:0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.599477:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762900.599486:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762900.599486:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762900.599525:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022185088 00000100:00100000:1.0:1789762900.599536:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022185088 00000100:00100000:3.0:1789762900.599551:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022185600:0@lo:400:kworker.0 00000100:00100000:2.0:1789762900.599559:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d09f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022185728:0@lo:400:kworker.0 00000100:00100000:0.0:1789762900.599597:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022185472 00000100:00100000:3.0:1789762900.599604:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762900.599605:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022185472 00000100:00100000:2.0:1789762900.599630:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762900.599631:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 339 00000100:00100000:1.0:1789762900.599648:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850da450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022185088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.599675:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022185216 00000100:00100000:3.0:1789762900.599682:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022185216 00000100:00100000:0.0:1789762900.599715:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022185600 00000100:00100000:0.0:1789762900.599724:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022185600 00000100:00100000:2.0:1789762900.599739:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 335 00000100:00100000:1.0:1789762900.599745:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850da450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022185088:12345-0@lo:400:kworker.0 Request processed in 99us (282us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762900.599757:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 140 00000100:00100000:1.0:1789762900.599768:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 339 00000100:00100000:2.0:1789762900.599768:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe72300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022185472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762900.599783:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:3.0:1789762900.599784:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d08380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022185216:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762900.599799:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022185600:12345-0@lo:400:kworker.0 00002000:00000024:3.0:1789762900.599828:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762900.599838:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762900.599845:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe72300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022185472:12345-0@lo:400:kworker.0 Request processed in 94us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762900.599856:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 335 00000100:00100000:2.0:1789762900.599889:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022185344 00000100:00100000:0.0:1789762900.599890:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022185600:12345-0@lo:400:kworker.0 Request processed in 93us (287us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762900.599897:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022185344 00000100:00100000:0.0:1789762900.599904:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:3.0:1789762900.599916:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d08380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022185216:12345-0@lo:13:osp-pre-1-0.0 Request processed in 134us (449us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762900.599931:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 140 00000100:00100000:1.0:1789762900.599951:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022185088:0@lo:400:kworker.0 00000100:00100000:2.0:1789762900.599958:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:2.0:1789762900.599973:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022185344:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762900.599990:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022185600:0@lo:400:kworker.0 00000100:00080000:0.0:1789762900.599991:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762900.599992:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022185728 00000100:00080000:0.0:1789762900.600001:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762900.600001:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022185728 00000100:00100000:2.0:1789762900.600031:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022185344:12345-0@lo:400:kworker.0 Request processed in 60us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762900.600042:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00100000:0.0:1789762900.600072:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022185472:0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.600078:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:1.0:1789762900.600084:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d09c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022185344:0@lo:400:kworker.0 00000100:00100000:3.0:1789762900.600139:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022185728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762900.600176:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfe71f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022185216:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762900.600222:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022185728:12345-0@lo:400:kworker.0 Request processed in 133us (609us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762900.600241:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:2.0:1789762900.600244:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d09f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022185728:0@lo:400:kworker.0 00010000:00010000:0.0:1789762900.919207:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762901.943290:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762902.967263:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762903.991360:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762905.015200:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762905.186677:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762905.186748:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762905.186750:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015096704 00000100:00100000:3.0:1789762905.186763:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015096704 00000100:00100000:2.0:1789762905.186781:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762905.186844:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 340 00000100:00100000:1.0:1789762905.186847:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015096832 00000100:00100000:2.0:1789762905.186851:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762905.186857:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015097088 00000100:00100000:1.0:1789762905.186862:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015096832 00000100:00100000:0.0:1789762905.186867:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015097088 00000100:00100000:3.0:1789762905.186867:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015096704:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762905.186944:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015096960 00000100:00100000:2.0:1789762905.186953:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015096960 00000100:00100000:0.0:1789762905.186960:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 338 00000100:00100000:3.0:1789762905.186960:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015096704:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 99us (293us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762905.186989:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 339 00000100:00100000:3.0:1789762905.186990:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 340 00000100:00100000:0.0:1789762905.186993:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015097088:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762905.187022:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015096832:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762905.187035:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 167 00000100:00100000:2.0:1789762905.187055:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015096960:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762905.187056:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015097088:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 80us (272us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762905.187126:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 338 00000100:00100000:1.0:1789762905.187165:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015096832:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 151us (414us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762905.187183:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762905.187184:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 339 00000100:00100000:2.0:1789762905.187196:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015096960:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 137us (344us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762905.187199:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762905.187232:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 167 00000100:00100000:2.0:1789762905.207248:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762905.207273:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfe71f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022185856:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762905.207335:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762905.207387:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022185856 00000100:00100000:3.0:1789762905.207395:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022185856 00000100:00080000:0.0:1789762905.207431:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762905.207467:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 141 00000100:00100000:3.0:1789762905.207476:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d09c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022185856:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762905.207502:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762905.207557:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d09c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022185856:12345-0@lo:13:osp-pre-0-0.0 Request processed in 83us (222us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762905.207563:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 141 00000100:00100000:2.0:1789762905.207637:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfe71f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022185856:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762905.719252:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfe71880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022186112:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762905.719261:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022186240:0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.719302:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86d09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022185984:0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.719363:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762905.719377:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762905.719380:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762905.719445:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022186240 00000100:00100000:3.0:1789762905.719457:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022186240 00000100:00100000:0.0:1789762905.719493:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022186752:0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.719501:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022186624:0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.719504:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022186496:0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.719536:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 340 00000100:00100000:3.0:1789762905.719553:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe73480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022186240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.719569:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762905.719589:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762905.719591:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022186368:0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.719657:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022186112 00000100:00100000:2.0:1789762905.719663:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762905.719666:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022186112 00000100:00100000:3.0:1789762905.719674:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe73480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022186240:12345-0@lo:400:kworker.0 Request processed in 119us (307us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762905.719680:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022186240:0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.719689:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 340 00000100:00100000:2.0:1789762905.719727:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762905.719733:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022186624 00000100:00100000:3.0:1789762905.719742:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022186624 00000100:00100000:1.0:1789762905.719743:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:0.0:1789762905.719755:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022186368 00000100:00100000:0.0:1789762905.719763:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022186368 00000100:00100000:1.0:1789762905.719764:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022186112:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762905.719780:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022186752 00000100:00100000:2.0:1789762905.719786:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022186752 00002000:00000024:1.0:1789762905.719809:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762905.719809:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 341 00000100:00100000:0.0:1789762905.719826:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:2.0:1789762905.719830:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022186496 00000100:00100000:3.0:1789762905.719831:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022186624:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.719835:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022186496 00000100:00100000:0.0:1789762905.719844:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022186368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.719875:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:1.0:1789762905.719888:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022186112:12345-0@lo:13:osp-pre-1-0.0 Request processed in 132us (518us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762905.719892:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022186752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.719899:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:3.0:1789762905.719912:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022186624:12345-0@lo:400:kworker.0 Request processed in 88us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762905.719913:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022186368:12345-0@lo:400:kworker.0 Request processed in 75us (254us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762905.719927:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:1.0:1789762905.719931:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022185984 00000100:00100000:1.0:1789762905.719939:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022185984 00000100:00100000:3.0:1789762905.719939:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 341 00000100:00100000:2.0:1789762905.719980:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022186752:12345-0@lo:400:kworker.0 Request processed in 89us (412us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762905.719988:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:1.0:1789762905.719993:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 341 00000100:00100000:2.0:1789762905.719998:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:0.0:1789762905.720004:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022186496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.720006:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850dbc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022185984:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762905.720046:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022186368:0@lo:400:kworker.0 00000100:00100000:2.0:1789762905.720059:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022186752:0@lo:400:kworker.0 00000100:00100000:1.0:1789762905.720077:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850dbc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022185984:12345-0@lo:400:kworker.0 Request processed in 72us (702us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762905.720122:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 341 00000100:00100000:1.0:1789762905.720194:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022186624:0@lo:400:kworker.0 00000100:00080000:3.0:1789762905.720222:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762905.720224:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86d09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022185984:0@lo:400:kworker.0 00000100:00080000:3.0:1789762905.720230:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762905.720231:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfe71880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022186112:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762905.720250:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022186496:0@lo:400:kworker.0 00000100:00100000:0.0:1789762905.720406:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022186496:12345-0@lo:400:kworker.0 Request processed in 406us (684us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762905.720415:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00010000:00010000:1.0:1789762906.040181:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762906.681369:0:16977:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:1.0:1789762906.681384:0:16977:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: gid, str: 500 00000100:00100000:1.0:1789762906.681391:0:16977:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:1.0:1789762906.681399:0:16977:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:1.0:1789762906.681403:0:16977:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: gid, str: 500 00000100:00100000:1.0:1789762906.681411:0:16977:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981d81d4ab00 rate 5 gen 0 00000100:00100000:1.0:1789762906.681418:0:16977:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:1.0:1789762906.681424:0:16977:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:1.0:1789762906.681428:0:16977:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: gid, str: 500 00000100:00100000:1.0:1789762906.681430:0:16977:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981d81d4a500 rate 5 gen 0 00000100:00100000:1.0:1789762906.692504:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.692546:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015097216 00000100:00100000:0.0:1789762906.692555:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015097216 00000100:00100000:0.0:1789762906.692626:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 327 00000100:00100000:0.0:1789762906.692639:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:15345:x1876702015097216:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:0.0:1789762906.692658:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762906.692685:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05225 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd735 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.692754:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b0522c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.692764:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b0522c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.692774:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b0522c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762906.692801:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762906.692805:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762906.692818:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b0522c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.692837:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05225 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd735 expref: 7 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.692849:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b0522c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd735 expref: 6 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.692855:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762906.692857:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762906.692860:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc600, rc 0) 00000100:00100000:0.0:1789762906.692913:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15345:x1876702015097216:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 278us (414us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762906.692919:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 327 00000100:00100000:3.0:1789762906.694041:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762906.694079:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015097472 00000100:00100000:1.0:1789762906.694115:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015097472 00000100:00100000:1.0:1789762906.694155:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 328 00000100:00100000:1.0:1789762906.694164:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15345:x1876702015097472:12345-192.168.203.48@tcp:36:mkdir.0 00010000:00010000:1.0:1789762906.694193:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762906.694208:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b0522c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd735 expref: 6 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.694228:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:1.0:1789762906.694255:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762906.694533:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b05233 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.694541:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b05233 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.694548:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b05233 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.694558:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d400/0x416193e012b0523a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.694584:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d400/0x416193e012b0523a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.694590:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d400/0x416193e012b0523a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762906.695504:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00010000:00010000:1.0:1789762906.695513:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b05233 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.695519:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b05233 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.695523:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b05233 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.695528:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762906.695531:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b05233 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.695537:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b05233 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.695544:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:1.0:1789762906.695546:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762906.695549:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b05233 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.695554:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b05233 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762906.695583:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b41d6600 reply state 00000000dc688875 transno 4294967578 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d400/0x416193e012b0523a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762906.695590:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762906.695640:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15345:x1876702015097472:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 1477us (1599us total) trans 4294967578 rc 0/0 00000100:00100000:1.0:1789762906.695646:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 328 00010000:00010000:3.0:1789762906.696169:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d400/0x416193e012b0523a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.696178:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d400/0x416193e012b0523a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.696183:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d400/0x416193e012b0523a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.696188:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762906.696190:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d400/0x416193e012b0523a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.696195:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d400/0x416193e012b0523a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.696204:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179000) 00010000:00010000:3.0:1789762906.696207:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762906.696209:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d400/0x416193e012b0523a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.696213:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d400/0x416193e012b0523a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762906.701917:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762906.701960:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015097600 00000100:00100000:1.0:1789762906.701967:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015097600 00000100:00100000:1.0:1789762906.702013:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 329 00000100:00100000:1.0:1789762906.702022:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15346:x1876702015097600:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762906.702037:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762906.702049:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05241 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd73c expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.702081:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05248 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.702205:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05248 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.702217:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05248 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762906.702236:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762906.702239:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762906.702250:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05248 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.702275:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05241 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd73c expref: 8 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.702288:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05248 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd73c expref: 7 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.702295:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179600) 00010000:00010000:1.0:1789762906.702298:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762906.702303:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4da00, rc 0) 00000100:00100000:1.0:1789762906.702350:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15346:x1876702015097600:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 327us (433us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762906.702357:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 329 00000100:00100000:1.0:1789762906.703376:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.703410:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015097856 00000100:00100000:0.0:1789762906.703421:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015097856 00000100:00100000:0.0:1789762906.703454:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 330 00000100:00100000:0.0:1789762906.703461:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15346:x1876702015097856:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762906.703469:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762906.703471:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1789762906.703482:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05248 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd73c expref: 7 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703496:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179600) 00010000:00010000:0.0:1789762906.703508:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762906.703517:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b0524f lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd743 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703604:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05256 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703612:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05256 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703618:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05256 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703627:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b0525d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703632:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b0525d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703637:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b0525d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703688:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05264 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703694:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05264 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703699:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05264 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762906.703705:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00010000:00010000:0.0:1789762906.703712:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05264 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703717:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05264 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703721:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05264 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703725:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762906.703727:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05264 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703731:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05264 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703737:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3500) 00010000:00010000:0.0:1789762906.703739:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762906.703741:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05264 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703754:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05264 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703767:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05256 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703771:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05256 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703774:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05256 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703778:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762906.703780:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05256 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703784:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05256 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703790:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3800) 00010000:00010000:0.0:1789762906.703792:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762906.703794:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05256 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703798:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05256 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703805:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b0525d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703809:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b0525d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703818:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b0525d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703822:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762906.703824:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b0525d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703828:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b0525d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703834:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762906.703835:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762906.703837:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b0525d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703841:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b0525d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703863:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b0524f lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd743 expref: 7 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703870:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3800) 00010000:00010000:0.0:1789762906.703872:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762906.703874:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b0524f lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd743 expref: 7 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.703886:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfd000, rc 0) 00000100:00100000:0.0:1789762906.703917:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15346:x1876702015097856:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 457us (543us total) trans 0 rc 301/301 00000100:00100000:0.0:1789762906.703931:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 330 00000100:00100000:0.0:1789762906.704740:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762906.704777:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015097984 00000100:00100000:1.0:1789762906.704783:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015097984 00000100:00100000:1.0:1789762906.704819:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 331 00000100:00100000:1.0:1789762906.704827:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15346:x1876702015097984:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762906.704836:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762906.704848:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0526b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd74a expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.704879:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05272 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.704890:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05272 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.704899:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05272 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762906.704920:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762906.704924:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762906.704937:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05272 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.704953:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0526b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd74a expref: 8 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.704965:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05272 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd74a expref: 7 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.704971:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:1.0:1789762906.704973:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762906.704975:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4cc00, rc 0) 00000100:00100000:1.0:1789762906.705009:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15346:x1876702015097984:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 182us (269us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762906.705014:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 331 00000100:00100000:1.0:1789762906.705978:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.706017:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015098240 00000100:00100000:0.0:1789762906.706024:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015098240 00000100:00100000:0.0:1789762906.706058:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 332 00000100:00100000:0.0:1789762906.706065:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15346:x1876702015098240:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762906.706073:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762906.706081:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b05279 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd751 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706162:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05280 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706170:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05280 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706176:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05280 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706185:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05287 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706190:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05287 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706194:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05287 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762906.706233:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00010000:00010000:0.0:1789762906.706278:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05280 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706290:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05280 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706294:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05280 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706298:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfcc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762906.706300:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05280 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706304:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05280 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706311:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:0.0:1789762906.706313:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762906.706315:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05280 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706319:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05280 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706327:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05287 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706331:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05287 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706334:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05287 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706337:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfce00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762906.706343:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05287 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706347:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05287 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706352:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762906.706354:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762906.706355:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05287 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706359:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05287 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706381:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b05279 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd751 expref: 9 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706387:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:0.0:1789762906.706389:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762906.706391:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b05279 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd751 expref: 9 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.706397:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfd200, rc 0) 00000100:00100000:0.0:1789762906.706430:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:15346:x1876702015098240:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 367us (454us total) trans 4294967579 rc 301/301 00000100:00100000:0.0:1789762906.706436:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 332 00000100:00100000:1.0:1789762906.707345:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.707385:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015098496 00000100:00100000:0.0:1789762906.707391:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015098496 00000100:00100000:0.0:1789762906.707423:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 333 00000100:00100000:0.0:1789762906.707431:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:15346:x1876702015098496:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:0.0:1789762906.707468:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05295 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.707475:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05295 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.707481:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05295 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762906.707602:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000352d18ad reply state 00000000bf246a0f transno 4294967580 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05295 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762906.707613:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762906.707619:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00000100:00100000:0.0:1789762906.707664:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15346:x1876702015098496:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 235us (320us total) trans 4294967580 rc 0/0 00000100:00100000:0.0:1789762906.707668:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 333 00010000:00010000:1.0:1789762906.708185:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05295 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.708194:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05295 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.708201:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05295 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.708205:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfde00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762906.708209:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05295 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.708215:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05295 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.708225:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3e00) 00010000:00010000:1.0:1789762906.708228:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789762906.708231:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05295 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.708237:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05295 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762906.708310:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.708348:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015098752 00000100:00100000:0.0:1789762906.708353:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015098752 00000100:00100000:0.0:1789762906.708383:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 105 00000100:00100000:0.0:1789762906.708389:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:15346:x1876702015098752:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:0.0:1789762906.708490:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15346:x1876702015098752:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 101us (184us total) trans 4294967581 rc 0/0 00000100:00100000:0.0:1789762906.708495:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 105 00000100:00100000:0.0:1789762906.712142:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762906.712160:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015098880 00000100:00100000:1.0:1789762906.712166:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015098880 00000100:00100000:1.0:1789762906.712201:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 334 00000100:00100000:1.0:1789762906.712209:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15347:x1876702015098880:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:1.0:1789762906.712221:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762906.712229:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0529c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd758 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712259:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052a3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712266:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052a3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712273:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052a3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712283:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052aa lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712289:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052aa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712294:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052aa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712330:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c000/0x416193e012b052b1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712335:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c000/0x416193e012b052b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712340:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c000/0x416193e012b052b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762906.712353:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00000004:00200000:1.0:1789762906.712355:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762906.712364:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c000/0x416193e012b052b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712369:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052a3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712374:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052a3 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712378:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052a3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712382:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762906.712384:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052a3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712390:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052a3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712398:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:1.0:1789762906.712401:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762906.712405:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052a3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712412:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052a3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712422:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052aa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712429:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052aa lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712436:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052aa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712442:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762906.712445:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052aa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712451:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052aa lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712459:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179500) 00010000:00010000:1.0:1789762906.712462:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762906.712465:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052aa lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712478:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052aa lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712503:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0529c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd758 expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712512:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c000/0x416193e012b052b1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd758 expref: 11 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.712517:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ec2179700) 00010000:00010000:1.0:1789762906.712520:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762906.712522:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4c000, rc 0) 00000100:00100000:1.0:1789762906.712555:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15347:x1876702015098880:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 348us (415us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762906.712560:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 334 00000100:00100000:2.0:1789762906.713436:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.713470:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015099136 00000100:00100000:0.0:1789762906.713477:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015099136 00000100:00100000:0.0:1789762906.713507:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 335 00000100:00100000:0.0:1789762906.713516:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15347:x1876702015099136:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:0.0:1789762906.713543:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762906.713555:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c000/0x416193e012b052b1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd758 expref: 11 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.713580:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ec2179700) 00010000:00010000:0.0:1789762906.713587:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762906.713598:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b052b8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.713605:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b052b8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.713610:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b052b8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762906.713702:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003725a888 reply state 000000005c3f2a2e transno 4294967582 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b052b8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762906.713708:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:0.0:1789762906.713712:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00000100:00100000:0.0:1789762906.713758:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15347:x1876702015099136:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 243us (328us total) trans 4294967582 rc 0/0 00000100:00100000:0.0:1789762906.713765:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 335 00010000:00010000:1.0:1789762906.714201:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b052b8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.714208:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b052b8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.714213:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b052b8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.714217:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762906.714219:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b052b8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.714224:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b052b8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.714233:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:1.0:1789762906.714235:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789762906.714237:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b052b8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.714242:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b052b8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762906.727726:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.727764:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015099264 00000100:00100000:0.0:1789762906.727772:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015099264 00000100:00100000:0.0:1789762906.727817:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 336 00000100:00100000:0.0:1789762906.727826:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15351:x1876702015099264:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762906.727839:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762906.727850:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b052bf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd75f expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.727897:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b052c6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.727908:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b052c6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.727916:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b052c6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.727927:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b052cd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.727932:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b052cd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.727938:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b052cd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728024:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b052d4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728034:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b052d4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728044:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b052d4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762906.728065:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00000004:00200000:0.0:1789762906.728067:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762906.728079:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b052d4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728119:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b052c6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728124:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b052c6 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728129:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b052c6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728134:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfcc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762906.728139:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b052c6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728147:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b052c6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728161:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:0.0:1789762906.728165:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762906.728170:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b052c6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728178:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b052c6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728191:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b052cd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728196:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b052cd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728200:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b052cd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728209:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762906.728212:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b052cd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728217:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b052cd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728224:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3800) 00010000:00010000:0.0:1789762906.728225:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762906.728227:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b052cd lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728232:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b052cd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728252:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b052bf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd75f expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728260:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b052d4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd75f expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.728265:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762906.728267:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789762906.728268:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc400, rc 0) 00000100:00100000:0.0:1789762906.728310:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15351:x1876702015099264:12345-192.168.203.48@tcp:101:dd.500 Request processed in 486us (589us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762906.728314:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 336 00000100:00100000:3.0:1789762906.729346:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762906.729797:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015099520 00000100:00100000:1.0:1789762906.729804:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015099520 00000100:00100000:1.0:1789762906.729841:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 337 00000100:00100000:1.0:1789762906.729853:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15351:x1876702015099520:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:1.0:1789762906.729864:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762906.729875:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052db lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd766 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.729961:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052e2 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.729973:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052e2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.729983:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052e2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.729999:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b052e9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730009:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b052e9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730023:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b052e9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789762906.730538:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x16:0x0], index 0: 0 00000004:00080000:1.0:1789762906.730550:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xe:0x0], index 1: 0 00010000:00010000:1.0:1789762906.730688:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c800/0x416193e012b052f0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730696:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c800/0x416193e012b052f0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730702:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c800/0x416193e012b052f0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730721:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052e2 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730726:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052e2 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730730:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052e2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730734:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762906.730736:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052e2 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730740:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052e2 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730746:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:1.0:1789762906.730748:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762906.730751:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052e2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730755:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b052e2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762906.730761:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000558e83ee reply state 00000000f95e29aa transno 4294967583 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b052e9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762906.730767:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789762906.730782:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b052db lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd766 expref: 15 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730790:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c800/0x416193e012b052f0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd766 expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762906.730798:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff981ec2179800) 00010000:00010000:1.0:1789762906.730802:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789762906.730804:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4c800, rc 0) 00000100:00100000:1.0:1789762906.730853:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15351:x1876702015099520:12345-192.168.203.48@tcp:101:dd.500 Request processed in 1002us (1508us total) trans 4294967583 rc 0/0 00000100:00100000:1.0:1789762906.730866:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 337 00010000:00010000:3.0:1789762906.731324:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b052e9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.731337:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b052e9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.731342:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b052e9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.731346:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4dc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762906.731349:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b052e9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.731355:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b052e9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.731368:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x77289514 (ffff981ec2179500) 00010000:00010000:3.0:1789762906.731370:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762906.731372:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b052e9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762906.731377:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b052e9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762906.733297:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.733354:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015099648 00000100:00100000:0.0:1789762906.733369:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015099648 00000100:00100000:0.0:1789762906.733431:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 187 00000100:00100000:0.0:1789762906.733444:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:15351:x1876702015099648:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762906.733454:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762906.733472:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b052fe lrc: 2/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd76d expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762906.733491:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x16:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789762906.733508:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=430s v=(0 0) 00010000:00010000:0.0:1789762906.733512:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762906.733520:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b052fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd76d expref: 6 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.733537:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b052fe lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd76d expref: 6 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.733548:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc600, rc 0) 00000100:00100000:0.0:1789762906.733604:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15351:x1876702015099648:12345-192.168.203.48@tcp:101:dd.500 Request processed in 160us (306us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762906.733616:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 187 00000100:00100000:1.0:1789762906.741357:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.741404:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015099776 00000100:00100000:0.0:1789762906.741409:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015099776 00002000:00100000:0.0:1789762906.741434:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x16:0x0] (0->1048575) req@ffff981ebfb1ea00 x1876702015099776/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/0 e 0 to 0 dl 1789762917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762906.741442:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1ea00 with x1876702015099776 ext(0->1048575) 00002000:00010000:0.0:1789762906.741446:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1ea00 00000100:00100000:0.0:1789762906.741465:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364636736525 time 364636843383 state 0x0 ref 1 00000100:00100000:0.0:1789762906.741471:0:14913:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 9: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 364636736525 time 364636843383 state 0x0 ref 1 00000100:00100000:0.0:1789762906.741483:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 3 deadline 430245391070 time 430045391070 state 0x1 ref 1 00000100:00100000:0.0:1789762906.741504:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 2 deadline 430245391070 time 430045413357 state 0x0 ref 1 00000100:00100000:0.0:1789762906.741514:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015099776:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762906.746814:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702015099776:12345-192.168.203.48@tcp:4:dd.500 Request processed in 5299us (5456us total) trans 4294968133 rc 0/0 00000100:00100000:0.0:1789762906.746822:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1126 00000100:00100000:0.0:1789762906.746829:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 2 deadline 430245391070 time 430045413357 state 0x2 ref 0 00002000:00100000:0.0:1789762906.746838:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x16:0x0] (0->1048575) req@ffff981ebfb1ea00 x1876702015099776/t4294968133(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/448 e 0 to 0 dl 1789762917 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762906.746845:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1ea00 with x1876702015099776 ext(0->1048575) 00002000:00010000:0.0:1789762906.746850:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1ea00 00000100:00100000:1.0:1789762906.751178:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762906.751223:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015099904 00000100:00100000:3.0:1789762906.751228:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015099904 00002000:00100000:3.0:1789762906.751249:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x240000400:0x16:0x0] (1048576->2097151) req@ffff981ebfb1ce00 x1876702015099904/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/0 e 0 to 0 dl 1789762917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762906.751257:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1ce00 with x1876702015099904 ext(1048576->2097151) 00002000:00010000:3.0:1789762906.751260:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1ce00 00000100:00100000:3.0:1789762906.751269:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 2 deadline 430245391070 time 430045413357 state 0x0 ref 1 00000100:00100000:3.0:1789762906.751278:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 2 deadline 430245413357 time 430045413357 state 0x1 ref 1 00000100:00100000:3.0:1789762906.751302:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 430245413357 time 430055209124 state 0x0 ref 1 00000100:00100000:3.0:1789762906.751314:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702015099904:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762906.753805:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702015099904:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2492us (2627us total) trans 4294968134 rc 0/0 00000100:00100000:0.0:1789762906.753813:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1127 00000100:00100000:0.0:1789762906.753819:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 430245413357 time 430055209124 state 0x2 ref 0 00002000:00100000:0.0:1789762906.753837:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x240000400:0x16:0x0] (1048576->2097151) req@ffff981ebfb1ce00 x1876702015099904/t4294968134(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/448 e 0 to 0 dl 1789762917 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762906.753845:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1ce00 with x1876702015099904 ext(1048576->2097151) 00002000:00010000:0.0:1789762906.753848:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1ce00 00000100:00100000:1.0:1789762906.758154:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762906.758220:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015100032 00000100:00100000:2.0:1789762906.758233:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015100032 00002000:00100000:2.0:1789762906.758289:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x16:0x0] (2097152->3145727) req@ffff981ebfb1dc00 x1876702015100032/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/0 e 0 to 0 dl 1789762917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762906.758304:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1dc00 with x1876702015100032 ext(2097152->3145727) 00002000:00010000:2.0:1789762906.758312:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1dc00 00000100:00100000:2.0:1789762906.758334:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 430245413357 time 430055209124 state 0x0 ref 1 00000100:00100000:2.0:1789762906.758353:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 430255209124 time 430055209124 state 0x1 ref 1 00000100:00100000:2.0:1789762906.758411:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430255209124 time 430062317192 state 0x0 ref 1 00000100:00100000:2.0:1789762906.758428:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702015100032:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762906.762206:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2364:x1876702015100032:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3774us (4047us total) trans 4294968135 rc 0/0 00000100:00100000:1.0:1789762906.762223:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1128 00000100:00100000:1.0:1789762906.762237:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430255209124 time 430062317192 state 0x2 ref 0 00002000:00100000:1.0:1789762906.762256:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x16:0x0] (2097152->3145727) req@ffff981ebfb1dc00 x1876702015100032/t4294968135(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/448 e 0 to 0 dl 1789762917 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762906.762268:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1dc00 with x1876702015100032 ext(2097152->3145727) 00002000:00010000:1.0:1789762906.762276:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1dc00 00000100:00100000:2.0:1789762906.766359:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.766410:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015100160 00000100:00100000:0.0:1789762906.766420:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015100160 00002000:00100000:0.0:1789762906.766457:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x16:0x0] (3145728->4194303) req@ffff981ebc49ce00 x1876702015100160/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/0 e 0 to 0 dl 1789762917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762906.766466:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49ce00 with x1876702015100160 ext(3145728->4194303) 00002000:00010000:0.0:1789762906.766470:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc49ce00 00000100:00100000:3.0:1789762906.766479:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.766483:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430255209124 time 430062317192 state 0x0 ref 1 00000100:00100000:0.0:1789762906.766503:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430262317192 time 430062317192 state 0x1 ref 1 00000100:00100000:1.0:1789762906.766606:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015100288 00000100:00100000:1.0:1789762906.766612:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015100288 00000100:00100000:1.0:1789762906.766642:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 188 00000100:00100000:1.0:1789762906.766649:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0+-99:2362:x1876702015100288:12345-192.168.203.48@tcp:8:dd.500 00010000:00080000:1.0:1789762906.766671:0:6649:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp t0 exp 0000000000000000 cur 430 last 0 00000020:00000080:1.0:1789762906.766695:0:6649:0:(genops.c:1385:class_connect()) connect: client f1bd28de-ee78-4d18-bad3-fa7cec968893, cookie 0x416193e012b05305 00002000:00100000:1.0:1789762906.766705:0:6649:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli f1bd28de-ee78-4d18-bad3-fa7cec968893/ffff981d8a645800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:1.0:1789762906.766714:0:6649:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:1.0:1789762906.766857:0:6649:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:1.0:1789762906.766879:0:6649:0:(import.c:65:import_set_state_nolock()) ffff981d8a646000 : changing import state from RECOVER to FULL 00000100:00100000:1.0:1789762906.766920:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2362:x1876702015100288:12345-192.168.203.48@tcp:8:dd.500 Request processed in 271us (444us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762906.766926:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 188 00000100:00100000:2.0:1789762906.767802:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.767850:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015100416 00000100:00100000:0.0:1789762906.767856:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015100416 00000100:00100000:0.0:1789762906.767895:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 189 00000100:00080000:3.0:1789762906.767897:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762906.767902:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15351:x1876702015100416:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762906.767912:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762906.767928:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001bb748ec/0x416193e012b0530c lrc: 2/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd774 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762906.767948:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789762906.767961:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=430s v=(0 0) 00010000:00010000:0.0:1789762906.767973:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762906.767980:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001bb748ec/0x416193e012b0530c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd774 expref: 7 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.767994:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001bb748ec/0x416193e012b0530c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd774 expref: 7 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762906.768004:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfce00, rc 0) 00000100:00100000:0.0:1789762906.768037:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15351:x1876702015100416:12345-192.168.203.48@tcp:101:dd.500 Request processed in 136us (236us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762906.768041:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 189 00000100:00100000:2.0:1789762906.775138:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762906.775187:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015100544 00000100:00100000:3.0:1789762906.775194:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015100544 00002000:00100000:3.0:1789762906.775215:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xe:0x0] (0->1048575) req@ffff981ebc49df80 x1876702015100544/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/0 e 0 to 0 dl 1789762917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762906.775224:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49df80 with x1876702015100544 ext(0->1048575) 00002000:00010000:3.0:1789762906.775227:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc49df80 00000100:00100000:3.0:1789762906.775237:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430262317192 time 430062317192 state 0x1 ref 2 00000100:00100000:2.0:1789762906.781905:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762906.781989:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015100672 00000100:00100000:0.0:1789762906.781996:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015100672 00002000:00100000:0.0:1789762906.782017:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0xe:0x0] (1048576->2097151) req@ffff981ebc49f800 x1876702015100672/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/0 e 0 to 0 dl 1789762917 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762906.782025:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49f800 with x1876702015100672 ext(1048576->2097151) 00002000:00010000:0.0:1789762906.782028:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc49f800 00000100:00100000:0.0:1789762906.782037:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430262317192 time 430062317192 state 0x1 ref 3 00000100:00100000:0.0:1789762906.958502:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430462396754 time 430262396754 state 0x1 ref 3 00000100:00100000:0.0:1789762906.958528:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702015100160:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762906.961730:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:2363:x1876702015100160:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3209us (195374us total) trans 4294968136 rc 0/0 00000100:00100000:0.0:1789762906.961741:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1129 00002000:00100000:0.0:1789762906.961760:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x240000400:0x16:0x0] (3145728->4194303) req@ffff981ebc49ce00 x1876702015100160/t4294968136(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/448 e 0 to 0 dl 1789762917 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762906.961768:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49ce00 with x1876702015100160 ext(3145728->4194303) 00002000:00010000:0.0:1789762906.961773:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc49ce00 00010000:00010000:1.0:1789762907.063211:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762907.158581:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430662464489 time 430462464489 state 0x1 ref 2 00000100:00100000:0.0:1789762907.158607:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702015100544:12345-192.168.203.48@tcp:4:dd.500 00000001:00100000:1.0:1789762907.169545:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000100:00100000:0.0:1789762907.171199:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702015100544:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12597us (396057us total) trans 4294967417 rc 0/0 00000100:00100000:0.0:1789762907.171214:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1130 00002000:00100000:0.0:1789762907.171230:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xe:0x0] (0->1048575) req@ffff981ebc49df80 x1876702015100544/t4294967417(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/448 e 0 to 0 dl 1789762917 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762907.171243:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49df80 with x1876702015100544 ext(0->1048575) 00002000:00010000:0.0:1789762907.171250:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc49df80 00000100:00100000:3.0:1789762907.173425:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.173474:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015100800 00000100:00100000:1.0:1789762907.173483:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015100800 00002000:00100000:1.0:1789762907.173514:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0xe:0x0] (2097152->3145727) req@ffff981d81ea4a80 x1876702015100800/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762907.173527:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81ea4a80 with x1876702015100800 ext(2097152->3145727) 00002000:00010000:1.0:1789762907.173530:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81ea4a80 00000100:00100000:1.0:1789762907.173546:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430662464489 time 430462464489 state 0x1 ref 2 00000100:00100000:2.0:1789762907.179870:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.179912:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015100928 00000100:00100000:1.0:1789762907.179918:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015100928 00002000:00100000:1.0:1789762907.179938:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0xe:0x0] (3145728->4194303) req@ffff981ebc49fb80 x1876702015100928/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762907.179946:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49fb80 with x1876702015100928 ext(3145728->4194303) 00002000:00010000:1.0:1789762907.179950:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc49fb80 00000100:00100000:1.0:1789762907.179959:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430662464489 time 430462464489 state 0x1 ref 3 00000100:00100000:2.0:1789762907.186751:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.186815:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015101056 00000100:00100000:1.0:1789762907.186822:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015101056 00002000:00100000:1.0:1789762907.186859:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x16:0x0] (4194304->5242879) req@ffff981ebc49d880 x1876702015101056/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00000100:00080000:3.0:1789762907.186864:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00100000:1.0:1789762907.186872:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49d880 with x1876702015101056 ext(4194304->5242879) 00002000:00010000:1.0:1789762907.186876:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc49d880 00000100:00100000:1.0:1789762907.186886:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430662464489 time 430462464489 state 0x1 ref 4 00000100:00100000:3.0:1789762907.193046:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.193102:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015101184 00000100:00100000:1.0:1789762907.193108:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015101184 00002000:00100000:1.0:1789762907.193128:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x16:0x0] (5242880->6291455) req@ffff981e851b8700 x1876702015101184/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762907.193141:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e851b8700 with x1876702015101184 ext(5242880->6291455) 00002000:00010000:1.0:1789762907.193144:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e851b8700 00000100:00100000:1.0:1789762907.193153:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430662464489 time 430462464489 state 0x1 ref 5 00000100:00100000:2.0:1789762907.199865:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762907.199913:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015101312 00000100:00100000:0.0:1789762907.199919:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015101312 00002000:00100000:0.0:1789762907.199938:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x16:0x0] (6291456->7340031) req@ffff981ebc49f100 x1876702015101312/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762907.199946:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49f100 with x1876702015101312 ext(6291456->7340031) 00002000:00010000:0.0:1789762907.199949:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc49f100 00000100:00100000:0.0:1789762907.199958:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430662464489 time 430462464489 state 0x1 ref 6 00000100:00100000:2.0:1789762907.206740:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762907.206792:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015101440 00000100:00100000:0.0:1789762907.206799:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015101440 00002000:00100000:0.0:1789762907.206828:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x16:0x0] (7340032->8388607) req@ffff981ebfdfc700 x1876702015101440/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762907.206835:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfc700 with x1876702015101440 ext(7340032->8388607) 00002000:00010000:0.0:1789762907.206838:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdfc700 00000100:00100000:0.0:1789762907.206847:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430662464489 time 430462464489 state 0x1 ref 7 00000100:00100000:3.0:1789762907.358663:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430862558764 time 430662558764 state 0x1 ref 7 00000100:00100000:3.0:1789762907.358688:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702015100672:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762907.361550:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702015100672:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2866us (579643us total) trans 4294967418 rc 0/0 00000100:00100000:0.0:1789762907.361574:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1131 00002000:00100000:0.0:1789762907.361585:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xe:0x0] (1048576->2097151) req@ffff981ebc49f800 x1876702015100672/t4294967418(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:687/0 lens 488/448 e 0 to 0 dl 1789762917 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762907.361593:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49f800 with x1876702015100672 ext(1048576->2097151) 00002000:00010000:0.0:1789762907.361599:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc49f800 00000100:00100000:2.0:1789762907.363837:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.363883:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015101568 00000100:00100000:1.0:1789762907.363891:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015101568 00002000:00100000:1.0:1789762907.363921:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xe:0x0] (4194304->5242879) req@ffff981ebfdfdf80 x1876702015101568/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762907.363929:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfdf80 with x1876702015101568 ext(4194304->5242879) 00002000:00010000:1.0:1789762907.363932:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdfdf80 00000100:00100000:1.0:1789762907.363945:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430862558764 time 430662558764 state 0x1 ref 7 00000100:00100000:3.0:1789762907.371142:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.371189:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015101696 00000100:00100000:1.0:1789762907.371194:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015101696 00002000:00100000:1.0:1789762907.371215:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xe:0x0] (5242880->6291455) req@ffff981e851b9c00 x1876702015101696/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762907.371223:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e851b9c00 with x1876702015101696 ext(5242880->6291455) 00002000:00010000:1.0:1789762907.371226:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e851b9c00 00000100:00100000:1.0:1789762907.371235:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 430862558764 time 430662558764 state 0x1 ref 8 00000100:00100000:0.0:1789762907.558741:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 431062633052 time 430862633052 state 0x1 ref 8 00000100:00100000:0.0:1789762907.558771:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:2363:x1876702015100800:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762907.562066:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:2363:x1876702015100800:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3305us (388639us total) trans 4294967419 rc 0/0 00000100:00100000:0.0:1789762907.562076:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1132 00002000:00100000:0.0:1789762907.562119:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xe:0x0] (2097152->3145727) req@ffff981d81ea4a80 x1876702015100800/t4294967419(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762907.562130:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81ea4a80 with x1876702015100800 ext(2097152->3145727) 00002000:00010000:0.0:1789762907.562138:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81ea4a80 00000100:00100000:3.0:1789762907.564267:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.564313:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015101824 00000100:00100000:1.0:1789762907.564320:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015101824 00002000:00100000:1.0:1789762907.564350:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0xe:0x0] (6291456->7340031) req@ffff981e851b8000 x1876702015101824/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762907.564363:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e851b8000 with x1876702015101824 ext(6291456->7340031) 00002000:00010000:1.0:1789762907.564365:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e851b8000 00000100:00100000:1.0:1789762907.564379:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 431062633052 time 430862633052 state 0x1 ref 8 00000100:00100000:2.0:1789762907.570659:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762907.570705:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.570719:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015101952 00000100:00100000:1.0:1789762907.570725:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015101952 00002000:00100000:1.0:1789762907.570753:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0xe:0x0] (7340032->8388607) req@ffff981ebfdfed80 x1876702015101952/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00000100:00100000:3.0:1789762907.570766:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015102208 00002000:00100000:1.0:1789762907.570767:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfed80 with x1876702015101952 ext(7340032->8388607) 00002000:00010000:1.0:1789762907.570774:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdfed80 00000100:00100000:3.0:1789762907.570774:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015102208 00000100:00100000:1.0:1789762907.570789:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 431062633052 time 430862633052 state 0x1 ref 9 00000100:00080000:0.0:1789762907.570829:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762907.570841:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 106 00000100:00100000:3.0:1789762907.570854:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7ad6450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15351:x1876702015102208:12345-192.168.203.48@tcp:35:dd.500 00000001:00080000:3.0:1789762907.571056:0:5808:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967584 00000100:00100000:3.0:1789762907.571153:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7ad6450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:15351:x1876702015102208:12345-192.168.203.48@tcp:35:dd.500 Request processed in 302us (448us total) trans 4294967585 rc 0/0 00000100:00100000:3.0:1789762907.571162:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 106 00000020:00080000:3.0:1789762907.571167:0:5808:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1789762907.613138:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762907.613190:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015102464 00000100:00100000:0.0:1789762907.613197:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015102464 00000100:00100000:0.0:1789762907.613251:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 338 00000100:00100000:0.0:1789762907.613263:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851baa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:15362:x1876702015102464:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762907.613281:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762907.613297:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05313 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd77b expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.613382:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b0531a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.613394:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b0531a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.613402:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b0531a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.613412:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05321 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.613417:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05321 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.613421:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05321 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762907.613820:0:6611:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xf:0x0], index 1: 0 00000004:00080000:0.0:1789762907.613825:0:6611:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x17:0x0], index 0: 0 00010000:00010000:0.0:1789762907.613961:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b05328 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.613984:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b05328 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.613993:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b05328 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.614028:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b0531a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.614037:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b0531a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.614044:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b0531a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.614049:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfcc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762907.614052:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b0531a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.614058:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b0531a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.614068:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762907.614070:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762907.614073:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b0531a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.614078:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b0531a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762907.614115:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008af14114 reply state 00000000eb2d927e transno 4294967586 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05321 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762907.614128:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762907.614151:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05313 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd77b expref: 19 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.614163:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b05328 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd77b expref: 18 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.614169:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x39:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762907.614171:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762907.614173:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc200, rc 0) 00000100:00100000:0.0:1789762907.614220:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851baa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:15362:x1876702015102464:12345-192.168.203.48@tcp:101:dd.0 Request processed in 960us (1087us total) trans 4294967586 rc 0/0 00000100:00100000:0.0:1789762907.614228:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 338 00010000:00010000:1.0:1789762907.615577:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05321 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.615585:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05321 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.615590:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05321 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.615593:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762907.615596:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05321 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.615600:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05321 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.615608:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x992ad74f (ffff981ea0aa3e00) 00010000:00010000:1.0:1789762907.615610:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762907.615612:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05321 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.615617:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05321 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762907.621875:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762907.621916:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015102592 00000100:00100000:0.0:1789762907.621922:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015102592 00000100:00100000:0.0:1789762907.621955:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 190 00000100:00100000:0.0:1789762907.621963:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:15364:x1876702015102592:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762907.621972:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762907.621984:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd782 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762907.622005:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789762907.622020:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=430s v=(0 0) 00010000:00010000:0.0:1789762907.622025:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762907.622036:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 18 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762907.622060:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762907.622076:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 18 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.622110:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfcc00, rc 0) 00000100:00100000:1.0:1789762907.622118:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015102720 00000100:00100000:1.0:1789762907.622124:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015102720 00000100:00100000:0.0:1789762907.622149:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:15364:x1876702015102592:12345-192.168.203.48@tcp:101:dd.0 Request processed in 185us (273us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762907.622162:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 190 00000100:00100000:1.0:1789762907.622169:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 191 00000100:00100000:1.0:1789762907.622181:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15363:x1876702015102720:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762907.622188:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762907.622197:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000be714456/0x416193e012b0533d lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd789 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.622223:0:6649:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 19 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.622229:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762907.622232:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789762907.622236:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000be714456/0x416193e012b0533d lrc: 4/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd789 expref: 19 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.622276:0:6649:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000000000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 19 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.622298:0:6649:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x70000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 19 pid: 6647 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.622314:0:6649:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6649:1876702022186880:192.168.203.48@tcp:104: 00000100:00100000:1.0:1789762907.622383:0:6649:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e90992c00 going to sleep for 11 seconds 00000100:00100000:1.0:1789762907.623152:0:6649:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6649:1876702022186880:192.168.203.48@tcp:104: 00000100:00100000:2.0:1789762907.623160:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:1.0:1789762907.623174:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000be714456/0x416193e012b0533d lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd789 expref: 19 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.623189:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4d000, rc 0) 00000100:00100000:0.0:1789762907.623196:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015102848 00000100:00100000:0.0:1789762907.623200:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015102848 00000100:00100000:1.0:1789762907.623225:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:15363:x1876702015102720:12345-192.168.203.48@tcp:101:dd.0 Request processed in 1047us (1168us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762907.623228:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 192 00000100:00100000:1.0:1789762907.623233:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 191 00000100:00100000:0.0:1789762907.623236:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:15362:x1876702015102848:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762907.623245:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762907.623255:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05344 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd790 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.623272:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789762907.623277:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(2 0) 00010000:00010000:0.0:1789762907.623282:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05344 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd790 expref: 20 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.623291:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05344 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd790 expref: 20 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.623299:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfd400, rc 0) 00000100:00100000:0.0:1789762907.623325:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:15362:x1876702015102848:12345-192.168.203.48@tcp:101:dd.0 Request processed in 90us (168us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762907.623336:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 192 00000100:00100000:2.0:1789762907.633254:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762907.633289:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015102976 00000100:00100000:3.0:1789762907.633295:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015102976 00000100:00100000:3.0:1789762907.633322:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 193 00000100:00100000:3.0:1789762907.633327:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:15370:x1876702015102976:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762907.633334:0:10916:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762907.633341:0:10916:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c2cc60cf/0x416193e012b0534b lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd797 expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.633355:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xf:0x0].0x0 since=0s v=(2 0) 00010000:00010000:3.0:1789762907.633358:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(3 0) 00010000:00010000:3.0:1789762907.633362:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c2cc60cf/0x416193e012b0534b lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 5 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd797 expref: 21 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.633370:0:10916:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c2cc60cf/0x416193e012b0534b lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 5 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd797 expref: 21 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.633382:0:10916:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d865a6200, rc 0) 00000100:00100000:3.0:1789762907.633408:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:15370:x1876702015102976:12345-192.168.203.48@tcp:101:dd.0 Request processed in 81us (155us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762907.633413:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 193 00000100:00100000:3.0:1789762907.636177:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.636222:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015103232 00000100:00100000:1.0:1789762907.636227:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015103232 00000100:00100000:1.0:1789762907.636256:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 194 00000100:00100000:1.0:1789762907.636263:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:15371:x1876702015103232:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762907.636270:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762907.636277:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000016074eb0/0x416193e012b05352 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7a5 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.636290:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xf:0x0].0x0 since=0s v=(3 0) 00010000:00010000:1.0:1789762907.636293:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(4 0) 00010000:00010000:1.0:1789762907.636297:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000016074eb0/0x416193e012b05352 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 6 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7a5 expref: 22 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.636305:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000016074eb0/0x416193e012b05352 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 6 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7a5 expref: 22 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.636312:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4de00, rc 0) 00000100:00100000:1.0:1789762907.636339:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:15371:x1876702015103232:12345-192.168.203.48@tcp:101:dd.0 Request processed in 77us (163us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762907.636344:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 194 00000100:00100000:2.0:1789762907.645834:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762907.645882:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015103488 00000100:00100000:0.0:1789762907.645889:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015103488 00000100:00100000:0.0:1789762907.645924:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 195 00000100:00100000:0.0:1789762907.645931:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15366:x1876702015103488:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762907.645941:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762907.645955:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7b3 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762907.645974:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x17:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789762907.645986:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=430s v=(0 0) 00010000:00010000:0.0:1789762907.645990:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762907.645998:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 19 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.646013:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 19 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.646024:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfd000, rc 0) 00000100:00100000:0.0:1789762907.646058:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:15366:x1876702015103488:12345-192.168.203.48@tcp:101:dd.0 Request processed in 128us (225us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762907.646063:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 195 00000100:00100000:2.0:1789762907.646609:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762907.646642:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015103104 00000100:00100000:3.0:1789762907.646648:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015103104 00000100:00100000:3.0:1789762907.646674:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 196 00000100:00100000:3.0:1789762907.646680:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:15369:x1876702015103104:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762907.646687:0:10916:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762907.646694:0:10916:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d2ddc2e3/0x416193e012b05360 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd79e expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.646708:0:10916:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 20 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.646715:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762907.646718:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789762907.646721:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d2ddc2e3/0x416193e012b05360 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd79e expref: 20 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.646739:0:10916:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 20 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.646747:0:10916:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 20 pid: 6881 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762907.646755:0:10916:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e851bb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10916:1876702022187008:192.168.203.48@tcp:104: 00000100:00100000:3.0:1789762907.646800:0:10916:0:(client.c:2715:ptlrpc_set_wait()) set ffff981eb464acc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789762907.647514:0:10916:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e851bb100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:10916:1876702022187008:192.168.203.48@tcp:104: 00010000:00010000:1.0:1789762907.647529:0:10916:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d2ddc2e3/0x416193e012b05360 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd79e expref: 20 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.647539:0:10916:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d865a7c00, rc 0) 00000100:00100000:1.0:1789762907.647577:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:15369:x1876702015103104:12345-192.168.203.48@tcp:101:dd.0 Request processed in 896us (967us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762907.647585:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 196 00000100:00100000:2.0:1789762907.648299:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762907.648324:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.648351:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015103616 00000100:00100000:2.0:1789762907.648351:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.648357:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015103616 00000100:00100000:0.0:1789762907.648378:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015103744 00000100:00100000:0.0:1789762907.648385:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015103744 00000100:00100000:1.0:1789762907.648388:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015103872 00000100:00100000:1.0:1789762907.648394:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015103872 00000100:00100000:3.0:1789762907.648401:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 197 00000100:00100000:2.0:1789762907.648411:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 198 00000100:00100000:3.0:1789762907.648415:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:15365:x1876702015103616:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762907.648424:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1789762907.648429:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:15367:x1876702015103744:12345-192.168.203.48@tcp:101:dd.0 00000100:00100000:0.0:1789762907.648430:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 199 00010000:00010000:3.0:1789762907.648435:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e502f589/0x416193e012b05367 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7ba expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.648446:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1789762907.648448:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:15376:x1876702015103872:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762907.648455:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xf:0x0].0x0 since=0s v=(4 0) 00010000:00010000:2.0:1789762907.648458:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b0536e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7c1 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.648464:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(5 0) 00010000:00010000:0.0:1789762907.648470:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762907.648475:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e502f589/0x416193e012b05367 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ba expref: 23 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.648484:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b05375 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7c8 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.648490:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789762907.648497:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e502f589/0x416193e012b05367 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 7 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ba expref: 23 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.648498:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762907.648508:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b0536e lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 5 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c1 expref: 23 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.648519:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d865a7400, rc 0) 00010000:00010000:0.0:1789762907.648524:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x17:0x0].0x0 since=0s v=(2 0) 00010000:00010000:0.0:1789762907.648531:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(3 0) 00010000:00010000:0.0:1789762907.648539:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b05375 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 5 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c8 expref: 23 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762907.648551:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15365:x1876702015103616:12345-192.168.203.48@tcp:101:dd.0 Request processed in 141us (251us total) trans 0 rc 0/0 00010000:00010000:0.0:1789762907.648555:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b05375 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 6 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c8 expref: 23 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.648565:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b0536e lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 6 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c1 expref: 23 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762907.648576:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 197 00010000:00010000:2.0:1789762907.648602:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e81482c00, rc 0) 00010000:00010000:0.0:1789762907.648603:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc000, rc 0) 00000100:00100000:2.0:1789762907.648636:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15367:x1876702015103744:12345-192.168.203.48@tcp:101:dd.0 Request processed in 213us (309us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762907.648645:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15376:x1876702015103872:12345-192.168.203.48@tcp:101:dd.0 Request processed in 200us (289us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762907.648652:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 198 00000100:00100000:0.0:1789762907.648668:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 199 00000100:00100000:2.0:1789762907.650216:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.650260:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015104000 00000100:00100000:1.0:1789762907.650265:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015104000 00000100:00100000:1.0:1789762907.650295:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 200 00000100:00100000:1.0:1789762907.650304:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15373:x1876702015104000:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762907.650314:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762907.650336:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008cdb5dbf/0x416193e012b0537c lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7cf expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.650353:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xf:0x0].0x0 since=0s v=(5 0) 00010000:00010000:1.0:1789762907.650356:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762907.650360:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008cdb5dbf/0x416193e012b0537c lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 8 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7cf expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.650368:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008cdb5dbf/0x416193e012b0537c lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 8 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7cf expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.650376:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4c600, rc 0) 00000100:00100000:1.0:1789762907.650404:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:15373:x1876702015104000:12345-192.168.203.48@tcp:101:dd.0 Request processed in 101us (188us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762907.650411:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 200 00000100:00100000:1.0:1789762907.650603:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762907.650637:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015104128 00000100:00100000:3.0:1789762907.650643:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015104128 00000100:00100000:3.0:1789762907.650668:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 201 00000100:00100000:3.0:1789762907.650674:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:15372:x1876702015104128:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762907.650682:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762907.650689:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000891aaaa3/0x416193e012b05383 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7d6 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.650702:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xf:0x0].0x0 since=0s v=(6 0) 00010000:00010000:3.0:1789762907.650706:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:3.0:1789762907.650710:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000891aaaa3/0x416193e012b05383 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7d6 expref: 25 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.650717:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000891aaaa3/0x416193e012b05383 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7d6 expref: 25 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.650724:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d865a6800, rc 0) 00000100:00100000:3.0:1789762907.650749:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:15372:x1876702015104128:12345-192.168.203.48@tcp:101:dd.0 Request processed in 76us (147us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762907.650754:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 201 00000100:00100000:2.0:1789762907.650892:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762907.650910:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762907.650944:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015104256 00000100:00100000:0.0:1789762907.650950:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015104256 00000100:00100000:1.0:1789762907.650961:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015103360 00000100:00100000:1.0:1789762907.650968:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015103360 00000100:00100000:0.0:1789762907.650993:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 202 00000100:00100000:0.0:1789762907.651008:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15377:x1876702015104256:12345-192.168.203.48@tcp:101:dd.0 00000100:00100000:1.0:1789762907.651009:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 203 00010000:00010000:0.0:1789762907.651021:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1789762907.651027:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15368:x1876702015103360:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762907.651033:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b0538a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7dd expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.651038:0:10916:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762907.651050:0:10916:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05391 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7ac expref: -99 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.651055:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x17:0x0].0x0 since=0s v=(3 0) 00010000:00010000:0.0:1789762907.651064:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(4 0) 00010000:00010000:0.0:1789762907.651079:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b0538a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 7 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7dd expref: 25 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.651119:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x17:0x0].0x0 since=0s v=(4 0) 00010000:00010000:1.0:1789762907.651126:0:10916:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(5 0) 00000100:00100000:2.0:1789762907.651131:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:1.0:1789762907.651132:0:10916:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05391 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 7 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ac expref: 25 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.651149:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b0538a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 8 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7dd expref: 25 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.651151:0:10916:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05391 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 8 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ac expref: 25 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762907.651153:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015104384 00000100:00100000:3.0:1789762907.651159:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015104384 00010000:00010000:0.0:1789762907.651180:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc800, rc 0) 00010000:00010000:1.0:1789762907.651182:0:10916:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4ce00, rc 0) 00000100:00100000:3.0:1789762907.651216:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 204 00000100:00100000:0.0:1789762907.651227:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:15377:x1876702015104256:12345-192.168.203.48@tcp:101:dd.0 Request processed in 218us (327us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762907.651228:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:15368:x1876702015103360:12345-192.168.203.48@tcp:101:dd.0 Request processed in 200us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762907.651228:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:15375:x1876702015104384:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762907.651244:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1789762907.651249:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 202 00010000:00010000:3.0:1789762907.651252:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006368f4ea/0x416193e012b05398 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7e4 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.651262:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 203 00010000:00010000:3.0:1789762907.651269:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x17:0x0].0x0 since=0s v=(5 0) 00010000:00010000:3.0:1789762907.651273:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(6 0) 00010000:00010000:3.0:1789762907.651279:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006368f4ea/0x416193e012b05398 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7e4 expref: 25 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.651288:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006368f4ea/0x416193e012b05398 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7e4 expref: 25 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.651296:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d865a7600, rc 0) 00000100:00100000:3.0:1789762907.651316:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:15375:x1876702015104384:12345-192.168.203.48@tcp:101:dd.0 Request processed in 94us (187us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762907.651320:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 204 00000100:00100000:2.0:1789762907.652241:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762907.652275:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015104512 00000100:00100000:3.0:1789762907.652279:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015104512 00000100:00100000:3.0:1789762907.652302:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 205 00000100:00100000:3.0:1789762907.652307:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:15374:x1876702015104512:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762907.652314:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762907.652319:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b0539f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7eb expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.652330:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x17:0x0].0x0 since=0s v=(6 0) 00010000:00010000:3.0:1789762907.652333:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:3.0:1789762907.652336:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b0539f lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7eb expref: 26 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.652343:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b0539f lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7eb expref: 26 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762907.652349:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d865a6c00, rc 0) 00000100:00100000:3.0:1789762907.652373:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:15374:x1876702015104512:12345-192.168.203.48@tcp:101:dd.0 Request processed in 66us (134us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762907.652377:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 205 00000100:00100000:0.0:1789762907.658355:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.658437:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015104640 00000100:00100000:1.0:1789762907.658443:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015104640 00002000:00100000:1.0:1789762907.658465:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x17:0x0] (0->1048575) req@ffff981e86d08a80 x1876702015104640/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762907.658474:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e86d08a80 with x1876702015104640 ext(0->1048575) 00010000:00010000:1.0:1789762907.658479:0:6652:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 26 pid: 6881 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.658489:0:6652:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 26 pid: 6881 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762907.658494:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff981e86d08a80 00000100:00100000:1.0:1789762907.658507:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 322968373672 time 322968319377 state 0x0 ref 1 00000100:00100000:1.0:1789762907.658514:0:6652:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 9: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 322968373672 time 322968319377 state 0x0 ref 1 00000100:00100000:1.0:1789762907.658524:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 322968419377 time 322968319377 state 0x1 ref 1 00000100:00100000:1.0:1789762907.658546:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 322968419377 time 430962455255 state 0x0 ref 1 00000100:00100000:1.0:1789762907.658555:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:2365:x1876702015104640:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762907.663046:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2365:x1876702015104640:12345-192.168.203.48@tcp:4:dd.0 Request processed in 4485us (4685us total) trans 4294968137 rc 0/0 00000100:00100000:1.0:1789762907.663060:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 6 00000100:00100000:1.0:1789762907.663071:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 322968419377 time 430962455255 state 0x2 ref 0 00002000:00100000:1.0:1789762907.663084:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x17:0x0] (0->1048575) req@ffff981e86d08a80 x1876702015104640/t4294968137(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762907.663109:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e86d08a80 with x1876702015104640 ext(0->1048575) 00010000:00010000:1.0:1789762907.663117:0:6652:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 28 pid: 6881 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.663128:0:6652:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 28 pid: 6881 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762907.663135:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff981e86d08a80 00000100:00100000:0.0:1789762907.664429:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762907.664475:0:5792:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015104768 00000100:00100000:2.0:1789762907.664483:0:5792:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015104768 00010000:00010000:2.0:1789762907.664511:0:5792:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 28 pid: 6881 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762907.664543:0:5792:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 62 00000100:00100000:2.0:1789762907.664550:0:5792:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2366:x1876702015104768:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762907.664572:0:5792:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1789762907.664585:0:5792:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:2.0:1789762907.664587:0:5792:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb mtime from disk: 0 -> 1789762907 00002000:00010000:2.0:1789762907.664590:0:5792:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb ctime from disk: 0 -> 1789762907 00010000:00010000:2.0:1789762907.664594:0:5792:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 28 pid: 6881 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664604:0:5792:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 28 pid: 6881 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664620:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b05359 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7b3 expref: 28 pid: 6881 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664632:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664636:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664649:0:5792:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d2ddc2e3/0x416193e012b05360 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd79e expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664656:0:5792:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d2ddc2e3/0x416193e012b05360 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd79e expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664663:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664664:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664668:0:5792:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b0536e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c1 expref: 27 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664673:0:5792:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b0536e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c1 expref: 27 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664678:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664680:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664683:0:5792:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b05375 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c8 expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664688:0:5792:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b05375 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c8 expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664692:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664693:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664696:0:5792:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b0538a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7dd expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664708:0:5792:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b0538a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7dd expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664713:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664714:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664718:0:5792:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05391 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ac expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664722:0:5792:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05391 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ac expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664727:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664728:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664731:0:5792:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006368f4ea/0x416193e012b05398 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7e4 expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664736:0:5792:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006368f4ea/0x416193e012b05398 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7e4 expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664740:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664741:0:5792:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(7 0) 00010000:00010000:2.0:1789762907.664745:0:5792:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b0539f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7eb expref: 27 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664749:0:5792:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b0539f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7eb expref: 27 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762907.664778:0:5792:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b0539f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7eb expref: 27 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762907.664788:0:5792:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdff100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187136:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762907.664854:0:5792:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000006368f4ea/0x416193e012b05398 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7e4 expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762907.664869:0:5792:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ea0fc4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187264:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762907.664896:0:5792:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05391 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ac expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762907.664905:0:5792:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ea0fc4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187392:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762907.664931:0:5792:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b0538a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7dd expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762907.664939:0:5792:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ea0fc5500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187520:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762907.664964:0:5792:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b05375 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c8 expref: 27 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762907.664971:0:5792:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ea0fc6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187648:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762907.665006:0:5792:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b0536e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c1 expref: 27 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762907.665023:0:5792:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ea0fc7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187776:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762907.665053:0:5792:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000d2ddc2e3/0x416193e012b05360 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd79e expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762907.665061:0:5792:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ea0fc4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187904:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762907.665077:0:5792:0:(client.c:2715:ptlrpc_set_wait()) set ffff981ebc2eff00 going to sleep for 11 seconds 00000100:00100000:2.0:1789762907.665682:0:5792:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdff100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187136:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762907.667417:0:5792:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ea0fc4a80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187264:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762907.667430:0:5792:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ea0fc4000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187392:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762907.667441:0:5792:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ea0fc5500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187520:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762907.667446:0:5792:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ea0fc6300 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187648:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762907.667451:0:5792:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ea0fc7480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187776:192.168.203.48@tcp:105: 00000100:00100000:2.0:1789762907.667456:0:5792:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ea0fc4700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5792:1876702022187904:192.168.203.48@tcp:105: 00010000:00010000:2.0:1789762907.667475:0:5792:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762907.667508:0:5792:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2366:x1876702015104768:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 2957us (3080us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762907.667518:0:5792:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 62 00000100:00100000:1.0:1789762907.707253:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762907.707301:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015104896 00000100:00100000:2.0:1789762907.707309:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015104896 00002000:00100000:2.0:1789762907.707338:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x17:0x0] (7340032->8388607) req@ffff981ebfb1f100 x1876702015104896/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762907.707363:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1f100 with x1876702015104896 ext(7340032->8388607) 00002000:00010000:2.0:1789762907.707368:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1f100 00000100:00100000:2.0:1789762907.707386:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400123578859 time 400123635211 state 0x0 ref 1 00000100:00100000:2.0:1789762907.707396:0:12582:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 9: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3400 gen 0 rate 20 depth 3 token 0 deadline 400123578859 time 400123635211 state 0x0 ref 1 00000100:00100000:2.0:1789762907.707425:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 3 deadline 431011424094 time 431011324094 state 0x1 ref 1 00000100:00100000:0.0:1789762907.707578:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431011424094 time 431011481036 state 0x0 ref 1 00000100:00100000:0.0:1789762907.707593:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2365:x1876702015104896:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762907.711174:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2365:x1876702015104896:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3581us (3922us total) trans 4294968138 rc 0/0 00000100:00100000:3.0:1789762907.711182:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1142 00000100:00100000:3.0:1789762907.711190:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431011424094 time 431011481036 state 0x2 ref 0 00002000:00100000:3.0:1789762907.711199:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x17:0x0] (7340032->8388607) req@ffff981ebfb1f100 x1876702015104896/t4294968138(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762907.711207:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1f100 with x1876702015104896 ext(7340032->8388607) 00002000:00010000:3.0:1789762907.711210:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1f100 00000100:00100000:1.0:1789762907.729160:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762907.729235:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015105024 00000100:00100000:2.0:1789762907.729244:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015105024 00002000:00100000:2.0:1789762907.729275:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x17:0x0] (3145728->4194303) req@ffff981ebfb1d880 x1876702015105024/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762907.729284:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1d880 with x1876702015105024 ext(3145728->4194303) 00002000:00010000:2.0:1789762907.729289:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1d880 00000100:00100000:2.0:1789762907.729300:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431011424094 time 431011481036 state 0x0 ref 1 00000100:00100000:2.0:1789762907.729330:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431011581036 time 431011481036 state 0x1 ref 1 00000100:00100000:2.0:1789762907.729353:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431011581036 time 431033259668 state 0x0 ref 1 00000100:00100000:2.0:1789762907.729366:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2366:x1876702015105024:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762907.732365:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762907.732415:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015105152 00000100:00100000:2.0:1789762907.732427:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015105152 00002000:00100000:2.0:1789762907.732448:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x17:0x0] (2097152->3145727) req@ffff981eb4bcd500 x1876702015105152/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762907.732456:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcd500 with x1876702015105152 ext(2097152->3145727) 00002000:00010000:2.0:1789762907.732459:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981eb4bcd500 00000100:00100000:2.0:1789762907.732496:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431033359668 time 431033259668 state 0x1 ref 2 00000100:00100000:0.0:1789762907.732517:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431033359668 time 431036419478 state 0x0 ref 2 00000100:00100000:0.0:1789762907.732536:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2365:x1876702015105152:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762907.732884:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2366:x1876702015105024:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3519us (3727us total) trans 4294968139 rc 0/0 00000100:00100000:1.0:1789762907.732892:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1143 00002000:00100000:1.0:1789762907.732899:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x17:0x0] (3145728->4194303) req@ffff981ebfb1d880 x1876702015105024/t4294968139(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762907.732907:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1d880 with x1876702015105024 ext(3145728->4194303) 00002000:00010000:1.0:1789762907.732911:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1d880 00000100:00100000:0.0:1789762907.733346:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762907.733394:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015105280 00000100:00100000:2.0:1789762907.733398:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015105280 00002000:00100000:2.0:1789762907.733415:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x17:0x0] (5242880->6291455) req@ffff981eb4bcf100 x1876702015105280/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762907.733422:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcf100 with x1876702015105280 ext(5242880->6291455) 00002000:00010000:2.0:1789762907.733425:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981eb4bcf100 00000100:00100000:2.0:1789762907.733450:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431036519478 time 431036419478 state 0x1 ref 2 00000100:00100000:2.0:1789762907.733468:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431036519478 time 431037377880 state 0x0 ref 2 00000100:00100000:2.0:1789762907.733481:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2366:x1876702015105280:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762907.735648:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2365:x1876702015105152:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3115us (3285us total) trans 4294968140 rc 0/0 00000100:00100000:3.0:1789762907.735656:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1144 00002000:00100000:3.0:1789762907.735663:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x17:0x0] (2097152->3145727) req@ffff981eb4bcd500 x1876702015105152/t4294968140(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762907.735670:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcd500 with x1876702015105152 ext(2097152->3145727) 00002000:00010000:3.0:1789762907.735674:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981eb4bcd500 00000100:00100000:3.0:1789762907.738217:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2366:x1876702015105280:12345-192.168.203.48@tcp:4:dd.0 Request processed in 4739us (4871us total) trans 4294968141 rc 0/0 00000100:00100000:3.0:1789762907.738226:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1145 00000100:00100000:3.0:1789762907.738236:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431036519478 time 431037377880 state 0x2 ref 0 00002000:00100000:3.0:1789762907.738253:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x17:0x0] (5242880->6291455) req@ffff981eb4bcf100 x1876702015105280/t4294968141(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762907.738264:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcf100 with x1876702015105280 ext(5242880->6291455) 00002000:00010000:3.0:1789762907.738268:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981eb4bcf100 00000100:00100000:1.0:1789762907.739361:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762907.739404:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015105408 00000100:00100000:2.0:1789762907.739409:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015105408 00002000:00100000:2.0:1789762907.739428:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x240000400:0x17:0x0] (1048576->2097151) req@ffff981e9f71d180 x1876702015105408/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762907.739435:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e9f71d180 with x1876702015105408 ext(1048576->2097151) 00002000:00010000:2.0:1789762907.739438:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e9f71d180 00000100:00100000:2.0:1789762907.739445:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431036519478 time 431037377880 state 0x0 ref 1 00000100:00100000:2.0:1789762907.739478:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431037477880 time 431037377880 state 0x1 ref 1 00000100:00100000:2.0:1789762907.739505:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431037477880 time 431043411529 state 0x0 ref 1 00000100:00100000:2.0:1789762907.739519:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2365:x1876702015105408:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762907.740606:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762907.740650:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015105536 00000100:00100000:2.0:1789762907.740654:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015105536 00002000:00100000:2.0:1789762907.740671:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x17:0x0] (6291456->7340031) req@ffff981eb4bcce00 x1876702015105536/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762907.740678:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcce00 with x1876702015105536 ext(6291456->7340031) 00002000:00010000:2.0:1789762907.740681:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981eb4bcce00 00000100:00100000:2.0:1789762907.740706:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431043511529 time 431043411529 state 0x1 ref 2 00000100:00100000:2.0:1789762907.740723:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431043511529 time 431044632851 state 0x0 ref 2 00000100:00100000:2.0:1789762907.740733:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2365:x1876702015105536:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762907.741936:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762907.741992:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015105664 00000100:00100000:3.0:1789762907.741998:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015105664 00002000:00100000:3.0:1789762907.742051:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x17:0x0] (4194304->5242879) req@ffff981ea0fc4700 x1876702015105664/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762907.742064:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ea0fc4700 with x1876702015105664 ext(4194304->5242879) 00002000:00010000:3.0:1789762907.742069:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ea0fc4700 00000100:00100000:3.0:1789762907.742156:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431044732851 time 431044632851 state 0x1 ref 3 00000100:00100000:2.0:1789762907.742194:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431044732851 time 431046099740 state 0x0 ref 3 00000100:00100000:2.0:1789762907.742211:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702015105664:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762907.744733:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2365:x1876702015105408:12345-192.168.203.48@tcp:4:dd.0 Request processed in 5216us (5372us total) trans 4294968142 rc 0/0 00000100:00100000:1.0:1789762907.744740:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1146 00002000:00100000:1.0:1789762907.744748:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x240000400:0x17:0x0] (1048576->2097151) req@ffff981e9f71d180 x1876702015105408/t4294968142(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762907.744755:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e9f71d180 with x1876702015105408 ext(1048576->2097151) 00002000:00010000:1.0:1789762907.744759:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e9f71d180 00000100:00100000:2.0:1789762907.747438:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2365:x1876702015105664:12345-192.168.203.48@tcp:4:dd.0 Request processed in 5226us (5501us total) trans 4294968143 rc 0/0 00000100:00100000:2.0:1789762907.747454:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1148 00002000:00100000:2.0:1789762907.747461:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x17:0x0] (4194304->5242879) req@ffff981ea0fc4700 x1876702015105664/t4294968143(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762907.747467:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ea0fc4700 with x1876702015105664 ext(4194304->5242879) 00002000:00010000:2.0:1789762907.747471:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ea0fc4700 00000100:00100000:0.0:1789762907.748952:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2365:x1876702015105536:12345-192.168.203.48@tcp:4:dd.0 Request processed in 8219us (8346us total) trans 4294968144 rc 0/0 00000100:00100000:0.0:1789762907.748962:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1147 00000100:00100000:0.0:1789762907.748978:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431044732851 time 431046099740 state 0x2 ref 0 00002000:00100000:0.0:1789762907.748990:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x17:0x0] (6291456->7340031) req@ffff981eb4bcce00 x1876702015105536/t4294968144(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762907.748999:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcce00 with x1876702015105536 ext(6291456->7340031) 00002000:00010000:0.0:1789762907.749004:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981eb4bcce00 00000100:00100000:3.0:1789762907.758776:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 431262680167 time 431062680167 state 0x1 ref 9 00000100:00100000:3.0:1789762907.758790:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2364:x1876702015100928:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762907.761446:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:2364:x1876702015100928:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2657us (581574us total) trans 4294967420 rc 0/0 00000100:00100000:2.0:1789762907.761458:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1133 00002000:00100000:2.0:1789762907.761465:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xe:0x0] (3145728->4194303) req@ffff981ebc49fb80 x1876702015100928/t4294967420(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762907.761473:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49fb80 with x1876702015100928 ext(3145728->4194303) 00002000:00010000:2.0:1789762907.761477:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc49fb80 00000100:00100000:1.0:1789762907.763653:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762907.763702:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015105792 00000100:00100000:0.0:1789762907.763710:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015105792 00002000:00100000:0.0:1789762907.763739:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xf:0x0] (2097152->3145727) req@ffff981e9f71c380 x1876702015105792/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762907.763749:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e9f71c380 with x1876702015105792 ext(2097152->3145727) 00010000:00010000:0.0:1789762907.763759:0:6654:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 26 pid: 6647 timeout: 530 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.763776:0:6654:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 26 pid: 6647 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762907.763784:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981e9f71c380 00000100:00100000:0.0:1789762907.763798:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 322968419377 time 430962455255 state 0x0 ref 1 00000100:00100000:0.0:1789762907.763807:0:6654:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 9: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 322968419377 time 430962455255 state 0x0 ref 1 00000100:00100000:0.0:1789762907.763822:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 430962555255 time 430962455255 state 0x1 ref 1 00000100:00100000:0.0:1789762907.763848:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 430962555255 time 431067755888 state 0x0 ref 1 00000100:00100000:0.0:1789762907.763874:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:2365:x1876702015105792:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762907.768910:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2365:x1876702015105792:12345-192.168.203.48@tcp:4:dd.0 Request processed in 5036us (5257us total) trans 4294967421 rc 0/0 00000100:00100000:0.0:1789762907.768921:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 7 00000100:00100000:0.0:1789762907.768930:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 430962555255 time 431067755888 state 0x2 ref 0 00002000:00100000:0.0:1789762907.768941:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xf:0x0] (2097152->3145727) req@ffff981e9f71c380 x1876702015105792/t4294967421(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762907.768950:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e9f71c380 with x1876702015105792 ext(2097152->3145727) 00010000:00010000:0.0:1789762907.768957:0:6654:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 28 pid: 6647 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762907.768971:0:6654:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 28 pid: 6647 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762907.768979:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981e9f71c380 00000100:00100000:2.0:1789762907.770349:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.770394:0:7595:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015105920 00000100:00100000:1.0:1789762907.770400:0:7595:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015105920 00010000:00010000:1.0:1789762907.770421:0:7595:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 28 pid: 6647 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.770446:0:7595:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 63 00000100:00100000:1.0:1789762907.770452:0:7595:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2366:x1876702015105920:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762907.770461:0:7595:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1789762907.770472:0:7595:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 0 -> 3145728 00002000:00010000:1.0:1789762907.770475:0:7595:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 0 -> 1789762907 00002000:00010000:1.0:1789762907.770477:0:7595:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 0 -> 1789762907 00010000:00010000:1.0:1789762907.770481:0:7595:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 28 pid: 6647 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770492:0:7595:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x50000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 28 pid: 6647 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770511:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d57ee577/0x416193e012b05336 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 10 type: EXT [0->18446744073709551615] (req 2097152->3145727) gid 0 flags: 0x44801400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd782 expref: 28 pid: 6647 timeout: 531 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770524:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=1s v=(7 0) 00010000:00010000:1.0:1789762907.770528:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770536:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000be714456/0x416193e012b0533d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd789 expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770543:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000be714456/0x416193e012b0533d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd789 expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770549:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770551:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770555:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05344 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd790 expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770580:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05344 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd790 expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770585:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770587:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770591:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c2cc60cf/0x416193e012b0534b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd797 expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770597:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c2cc60cf/0x416193e012b0534b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd797 expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770602:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770603:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770607:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000016074eb0/0x416193e012b05352 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7a5 expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770612:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000016074eb0/0x416193e012b05352 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7a5 expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770616:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770618:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770621:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e502f589/0x416193e012b05367 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ba expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770625:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e502f589/0x416193e012b05367 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ba expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770629:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770631:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770634:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000008cdb5dbf/0x416193e012b0537c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7cf expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770639:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008cdb5dbf/0x416193e012b0537c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7cf expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770644:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770646:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762907.770649:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000891aaaa3/0x416193e012b05383 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7d6 expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770654:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000891aaaa3/0x416193e012b05383 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7d6 expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762907.770679:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000891aaaa3/0x416193e012b05383 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7d6 expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.770689:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e9f71ed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188032:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762907.770764:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000008cdb5dbf/0x416193e012b0537c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7cf expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.770777:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e9f71ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188160:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762907.770816:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e502f589/0x416193e012b05367 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ba expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.770828:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e9f71c000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188288:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762907.770873:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000016074eb0/0x416193e012b05352 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7a5 expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.770889:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e9f71f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188416:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762907.770914:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c2cc60cf/0x416193e012b0534b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd797 expref: 27 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.770921:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e9f71d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188544:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762907.770943:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05344 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd790 expref: 27 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.770952:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc434700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188672:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762907.770990:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000be714456/0x416193e012b0533d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd789 expref: 27 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762907.771003:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc435880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188800:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762907.771032:0:7595:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e90992c00 going to sleep for 11 seconds 00000100:00100000:1.0:1789762907.771642:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e9f71ed80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188032:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762907.783418:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e9f71ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188160:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762907.783438:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e9f71c000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188288:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762907.783449:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e9f71f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188416:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762907.783459:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e9f71d880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188544:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762907.783468:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc434700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188672:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762907.783474:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc435880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022188800:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762907.783514:0:7595:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762907.783571:0:7595:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2366:x1876702015105920:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 13117us (13222us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762907.783580:0:7595:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 63 00000100:00100000:0.0:1789762907.812732:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762907.812780:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015106048 00000100:00100000:3.0:1789762907.812788:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015106048 00002000:00100000:3.0:1789762907.812825:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0xf:0x0] (1048576->2097151) req@ffff981eb4bcca80 x1876702015106048/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762907.812834:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcca80 with x1876702015106048 ext(1048576->2097151) 00002000:00010000:3.0:1789762907.812838:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eb4bcca80 00000100:00100000:3.0:1789762907.812853:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431044732851 time 431046099740 state 0x0 ref 1 00000100:00100000:3.0:1789762907.812888:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431046199740 time 431046099740 state 0x1 ref 1 00000100:00100000:3.0:1789762907.812910:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431046199740 time 431116818700 state 0x0 ref 1 00000100:00100000:3.0:1789762907.812920:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2364:x1876702015106048:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762907.815845:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2364:x1876702015106048:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2926us (3118us total) trans 4294967422 rc 0/0 00000100:00100000:3.0:1789762907.815852:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1149 00000100:00100000:3.0:1789762907.815858:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431046199740 time 431116818700 state 0x2 ref 0 00002000:00100000:3.0:1789762907.815866:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0xf:0x0] (1048576->2097151) req@ffff981eb4bcca80 x1876702015106048/t4294967422(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762907.815873:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcca80 with x1876702015106048 ext(1048576->2097151) 00002000:00010000:3.0:1789762907.815876:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eb4bcca80 00000100:00100000:0.0:1789762907.818880:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.818927:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015106176 00000100:00100000:1.0:1789762907.818931:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015106176 00002000:00100000:1.0:1789762907.818953:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0xf:0x0] (7340032->8388607) req@ffff981eb4bcd880 x1876702015106176/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762907.818961:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcd880 with x1876702015106176 ext(7340032->8388607) 00002000:00010000:1.0:1789762907.818964:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eb4bcd880 00000100:00100000:1.0:1789762907.818981:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431046199740 time 431116818700 state 0x0 ref 1 00000100:00100000:1.0:1789762907.819005:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431116918700 time 431116818700 state 0x1 ref 1 00000100:00100000:1.0:1789762907.819027:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431116918700 time 431122933739 state 0x0 ref 1 00000100:00100000:1.0:1789762907.819039:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2366:x1876702015106176:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762907.821203:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2366:x1876702015106176:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2163us (2321us total) trans 4294967423 rc 0/0 00000100:00100000:1.0:1789762907.821213:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1150 00000100:00100000:1.0:1789762907.821221:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431116918700 time 431122933739 state 0x2 ref 0 00002000:00100000:1.0:1789762907.821233:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0xf:0x0] (7340032->8388607) req@ffff981eb4bcd880 x1876702015106176/t4294967423(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762907.821245:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcd880 with x1876702015106176 ext(7340032->8388607) 00002000:00010000:1.0:1789762907.821249:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eb4bcd880 00000100:00100000:2.0:1789762907.833901:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762907.833977:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015106304 00000100:00100000:1.0:1789762907.833983:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015106304 00002000:00100000:1.0:1789762907.834004:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xf:0x0] (0->1048575) req@ffff981ea0fc6300 x1876702015106304/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762907.834012:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ea0fc6300 with x1876702015106304 ext(0->1048575) 00002000:00010000:1.0:1789762907.834016:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ea0fc6300 00000100:00100000:1.0:1789762907.834024:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431116918700 time 431122933739 state 0x0 ref 1 00000100:00100000:1.0:1789762907.834059:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431123033739 time 431122933739 state 0x1 ref 1 00000100:00100000:1.0:1789762907.834111:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431123033739 time 431137990912 state 0x0 ref 1 00000100:00100000:1.0:1789762907.834125:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:2365:x1876702015106304:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762907.837114:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2365:x1876702015106304:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2991us (3212us total) trans 4294967424 rc 0/0 00000100:00100000:1.0:1789762907.837121:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1151 00000100:00100000:1.0:1789762907.837127:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431123033739 time 431137990912 state 0x2 ref 0 00002000:00100000:1.0:1789762907.837137:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xf:0x0] (0->1048575) req@ffff981ea0fc6300 x1876702015106304/t4294967424(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762907.837149:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ea0fc6300 with x1876702015106304 ext(0->1048575) 00002000:00010000:1.0:1789762907.837152:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ea0fc6300 00000100:00100000:0.0:1789762907.839183:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762907.839222:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015106432 00000100:00100000:3.0:1789762907.839228:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015106432 00002000:00100000:3.0:1789762907.839246:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xf:0x0] (5242880->6291455) req@ffff981eb4bcea00 x1876702015106432/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762907.839254:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcea00 with x1876702015106432 ext(5242880->6291455) 00002000:00010000:3.0:1789762907.839258:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eb4bcea00 00000100:00100000:3.0:1789762907.839266:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431123033739 time 431137990912 state 0x0 ref 1 00000100:00100000:3.0:1789762907.839291:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431138090912 time 431137990912 state 0x1 ref 1 00000100:00100000:3.0:1789762907.839309:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431138090912 time 431143218291 state 0x0 ref 1 00000100:00100000:3.0:1789762907.839318:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2366:x1876702015106432:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762907.842352:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762907.842392:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015106560 00000100:00100000:0.0:1789762907.842398:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015106560 00002000:00100000:0.0:1789762907.842420:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0xf:0x0] (4194304->5242879) req@ffff981ea0fc5500 x1876702015106560/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762907.842427:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ea0fc5500 with x1876702015106560 ext(4194304->5242879) 00002000:00010000:0.0:1789762907.842437:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ea0fc5500 00000100:00100000:0.0:1789762907.842470:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431143318291 time 431143218291 state 0x1 ref 2 00000100:00100000:0.0:1789762907.842493:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431143318291 time 431146402412 state 0x0 ref 2 00000100:00100000:0.0:1789762907.842506:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2363:x1876702015106560:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762907.842574:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2366:x1876702015106432:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3255us (3389us total) trans 4294967425 rc 0/0 00000100:00100000:2.0:1789762907.842582:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1152 00002000:00100000:2.0:1789762907.842590:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0xf:0x0] (5242880->6291455) req@ffff981eb4bcea00 x1876702015106432/t4294967425(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762907.842599:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bcea00 with x1876702015106432 ext(5242880->6291455) 00002000:00010000:2.0:1789762907.842604:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eb4bcea00 00000100:00100000:2.0:1789762907.844303:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762907.844342:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015106688 00000100:00100000:3.0:1789762907.844460:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015106688 00002000:00100000:3.0:1789762907.844478:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0xf:0x0] (3145728->4194303) req@ffff981ea0fc4000 x1876702015106688/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762907.844487:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ea0fc4000 with x1876702015106688 ext(3145728->4194303) 00002000:00010000:3.0:1789762907.844490:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ea0fc4000 00000100:00100000:3.0:1789762907.844617:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431146502412 time 431146402412 state 0x1 ref 2 00000100:00100000:0.0:1789762907.844634:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431146502412 time 431148538436 state 0x0 ref 2 00000100:00100000:0.0:1789762907.844649:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2364:x1876702015106688:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762907.846700:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762907.847026:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762907.847052:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015107072 00000100:00100000:0.0:1789762907.847059:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015107072 00000100:00100000:0.0:1789762907.847120:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 107 00000100:00100000:0.0:1789762907.847127:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15372:x1876702015107072:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:3.0:1789762907.847148:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015106816 00000100:00100000:3.0:1789762907.847153:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015106816 00002000:00100000:3.0:1789762907.847172:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xf:0x0] (6291456->7340031) req@ffff981eb4bce680 x1876702015106816/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/0 e 0 to 0 dl 1789762918 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762907.847179:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bce680 with x1876702015106816 ext(6291456->7340031) 00002000:00010000:3.0:1789762907.847183:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eb4bce680 00000100:00100000:3.0:1789762907.847209:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431148638436 time 431148538436 state 0x1 ref 3 00000100:00100000:2.0:1789762907.847236:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431148638436 time 431151141512 state 0x0 ref 3 00000100:00100000:2.0:1789762907.847250:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702015106816:12345-192.168.203.48@tcp:4:dd.0 00000001:00080000:0.0:1789762907.847314:0:11718:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967587 00000100:00100000:0.0:1789762907.847389:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:15372:x1876702015107072:12345-192.168.203.48@tcp:35:dd.0 Request processed in 262us (364us total) trans 4294967588 rc 0/0 00000100:00100000:0.0:1789762907.847396:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 107 00000020:00080000:0.0:1789762907.847399:0:11718:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1789762907.849221:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2363:x1876702015106560:12345-192.168.203.48@tcp:4:dd.0 Request processed in 6717us (6869us total) trans 4294967426 rc 0/0 00000100:00100000:0.0:1789762907.849228:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1153 00002000:00100000:0.0:1789762907.849234:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0xf:0x0] (4194304->5242879) req@ffff981ea0fc5500 x1876702015106560/t4294967426(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762907.849252:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ea0fc5500 with x1876702015106560 ext(4194304->5242879) 00002000:00010000:0.0:1789762907.849256:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ea0fc5500 00000100:00100000:0.0:1789762907.850137:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2365:x1876702015106816:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2890us (3438us total) trans 4294967427 rc 0/0 00000100:00100000:0.0:1789762907.850143:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1155 00002000:00100000:0.0:1789762907.850149:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0xf:0x0] (6291456->7340031) req@ffff981eb4bce680 x1876702015106816/t4294967427(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762907.850155:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eb4bce680 with x1876702015106816 ext(6291456->7340031) 00002000:00010000:0.0:1789762907.850158:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eb4bce680 00000100:00100000:0.0:1789762907.851042:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2364:x1876702015106688:12345-192.168.203.48@tcp:4:dd.0 Request processed in 6396us (6739us total) trans 4294967428 rc 0/0 00000100:00100000:0.0:1789762907.851049:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1154 00000100:00100000:0.0:1789762907.851056:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431148638436 time 431151141512 state 0x2 ref 0 00002000:00100000:0.0:1789762907.851064:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xf:0x0] (3145728->4194303) req@ffff981ea0fc4000 x1876702015106688/t4294967428(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762907.851072:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ea0fc4000 with x1876702015106688 ext(3145728->4194303) 00002000:00010000:0.0:1789762907.851075:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ea0fc4000 00000100:00100000:1.0:1789762907.958875:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 431462770357 time 431262770357 state 0x1 ref 8 00000100:00100000:1.0:1789762907.958900:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2363:x1876702015101056:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762907.961885:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2363:x1876702015101056:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2991us (775133us total) trans 4294968145 rc 0/0 00000100:00100000:1.0:1789762907.961893:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1134 00002000:00100000:1.0:1789762907.961904:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x16:0x0] (4194304->5242879) req@ffff981ebc49d880 x1876702015101056/t4294968145(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762907.961912:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49d880 with x1876702015101056 ext(4194304->5242879) 00002000:00010000:1.0:1789762907.961917:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc49d880 00010000:00010000:2.0:1789762908.087209:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762908.158979:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 431662875695 time 431462875695 state 0x1 ref 7 00000100:00100000:2.0:1789762908.159003:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2364:x1876702015101184:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762908.161992:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:2364:x1876702015101184:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2995us (968943us total) trans 4294968146 rc 0/0 00000100:00100000:3.0:1789762908.162001:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1135 00002000:00100000:3.0:1789762908.162011:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x16:0x0] (5242880->6291455) req@ffff981e851b8700 x1876702015101184/t4294968146(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762908.162027:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e851b8700 with x1876702015101184 ext(5242880->6291455) 00002000:00010000:3.0:1789762908.162033:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e851b8700 00000001:00080000:0.0:1789762908.287327:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967588 is committed 00000100:00100000:3.0:1789762908.359045:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 431862939594 time 431662939594 state 0x1 ref 6 00000100:00100000:3.0:1789762908.359076:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2364:x1876702015101312:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762908.362582:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:2364:x1876702015101312:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3517us (1162711us total) trans 4294968147 rc 0/0 00000100:00100000:3.0:1789762908.362595:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1136 00002000:00100000:3.0:1789762908.362607:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x16:0x0] (6291456->7340031) req@ffff981ebc49f100 x1876702015101312/t4294968147(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762908.362630:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc49f100 with x1876702015101312 ext(6291456->7340031) 00002000:00010000:3.0:1789762908.362636:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebc49f100 00000100:00100000:2.0:1789762908.559142:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 432063037072 time 431863037072 state 0x1 ref 5 00000100:00100000:2.0:1789762908.559165:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:2363:x1876702015101440:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762908.561739:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+45:2363:x1876702015101440:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2580us (1354997us total) trans 4294968148 rc 0/0 00000100:00100000:1.0:1789762908.561747:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1137 00002000:00100000:1.0:1789762908.561757:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x16:0x0] (7340032->8388607) req@ffff981ebfdfc700 x1876702015101440/t4294968148(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762908.561765:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfc700 with x1876702015101440 ext(7340032->8388607) 00002000:00010000:1.0:1789762908.561770:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdfc700 00000100:00100000:3.0:1789762908.759211:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 432263107134 time 432063107134 state 0x1 ref 4 00000100:00100000:3.0:1789762908.759251:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2364:x1876702015101568:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:0.0:1789762908.759271:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762908.762255:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2364:x1876702015101568:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3028us (1398419us total) trans 4294967429 rc 0/0 00000100:00100000:3.0:1789762908.762264:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1138 00002000:00100000:3.0:1789762908.762274:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0xe:0x0] (4194304->5242879) req@ffff981ebfdfdf80 x1876702015101568/t4294967429(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762908.762281:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfdf80 with x1876702015101568 ext(4194304->5242879) 00002000:00010000:3.0:1789762908.762288:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdfdf80 00000100:00100000:3.0:1789762908.959282:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 432463176040 time 432263176040 state 0x1 ref 3 00000100:00100000:3.0:1789762908.959308:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2364:x1876702015101696:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762908.962197:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:2364:x1876702015101696:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2898us (1591085us total) trans 4294967430 rc 0/0 00000100:00100000:3.0:1789762908.962205:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1139 00002000:00100000:3.0:1789762908.962215:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0xe:0x0] (5242880->6291455) req@ffff981e851b9c00 x1876702015101696/t4294967430(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762908.962224:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e851b9c00 with x1876702015101696 ext(5242880->6291455) 00002000:00010000:3.0:1789762908.962229:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e851b9c00 00010000:00010000:0.0:1789762909.111184:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762909.159394:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 432663286587 time 432463286587 state 0x1 ref 2 00000100:00100000:0.0:1789762909.159444:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2364:x1876702015101824:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762909.162613:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+44:2364:x1876702015101824:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3175us (1598343us total) trans 4294967431 rc 0/0 00000100:00100000:2.0:1789762909.162626:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1140 00002000:00100000:2.0:1789762909.162639:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0xe:0x0] (6291456->7340031) req@ffff981e851b8000 x1876702015101824/t4294967431(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762909.162648:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e851b8000 with x1876702015101824 ext(6291456->7340031) 00002000:00010000:2.0:1789762909.162654:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981e851b8000 00000100:00100000:0.0:1789762909.359533:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 432663286587 time 432663427540 state 0x0 ref 1 00000100:00100000:0.0:1789762909.359573:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+43:2364:x1876702015101952:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:0.0:1789762909.365820:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+45:2364:x1876702015101952:12345-192.168.203.48@tcp:4:dd.500 Request processed in 6268us (1795160us total) trans 4294967432 rc 0/0 00000100:00100000:0.0:1789762909.365830:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1141 00000100:00100000:0.0:1789762909.365837:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 432663286587 time 432663427540 state 0x2 ref 0 00002000:00100000:0.0:1789762909.365850:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0xe:0x0] (7340032->8388607) req@ffff981ebfdfed80 x1876702015101952/t4294967432(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:688/0 lens 488/448 e 0 to 0 dl 1789762918 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762909.365858:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdfed80 with x1876702015101952 ext(7340032->8388607) 00002000:00010000:0.0:1789762909.365864:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdfed80 00000100:00100000:2.0:1789762909.402859:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.402921:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015107200 00000100:00100000:1.0:1789762909.402931:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015107200 00000100:00080000:0.0:1789762909.402993:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762909.403022:0:6664:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 30 00000100:00100000:1.0:1789762909.403044:0:6664:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+45:6050:x1876702015107200:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762909.403061:0:6664:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 8 locks, starting at 0 00002000:00010000:1.0:1789762909.403079:0:6664:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x16:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:1.0:1789762909.403082:0:6664:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x16:0x0] updating lvb mtime from disk: 0 -> 1789762907 00002000:00010000:1.0:1789762909.403084:0:6664:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x16:0x0] updating lvb ctime from disk: 0 -> 1789762907 00010000:00010000:1.0:1789762909.403122:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b052fe lrc: 0/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd76d expref: 45 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762909.403145:0:6664:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb size from disk: 1048576 -> 8388608 00010000:00010000:1.0:1789762909.403150:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b0539f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 9 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7eb expref: 44 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.403159:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b05375 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c8 expref: 43 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.403168:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b0536e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 7 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7c1 expref: 42 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.403176:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006368f4ea/0x416193e012b05398 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 6 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7e4 expref: 41 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.403204:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05391 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 5 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ac expref: 40 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.403214:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d2ddc2e3/0x416193e012b05360 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd79e expref: 39 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.403228:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ceb8603d/0x416193e012b0538a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7dd expref: 38 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.403237:0:6664:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762909.403287:0:6664:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:6050:x1876702015107200:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 248us (434us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762909.403294:0:6664:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 30 00000100:00100000:3.0:1789762909.403326:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762909.403372:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015107328 00000100:00100000:0.0:1789762909.403388:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015107328 00000100:00100000:0.0:1789762909.403433:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 31 00000100:00100000:0.0:1789762909.403450:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86f01c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+45:5592:x1876702015107328:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762909.403460:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 8 locks, starting at 0 00002000:00010000:0.0:1789762909.403473:0:5790:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:0.0:1789762909.403476:0:5790:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb mtime from disk: 0 -> 1789762907 00002000:00010000:0.0:1789762909.403479:0:5790:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xe:0x0] updating lvb ctime from disk: 0 -> 1789762907 00010000:00010000:0.0:1789762909.403503:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001bb748ec/0x416193e012b0530c lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd774 expref: 45 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762909.403524:0:5790:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 3145728 -> 8388608 00002000:00010000:0.0:1789762909.403526:0:5790:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 1789762907 -> 1789762908 00002000:00010000:0.0:1789762909.403528:0:5790:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 1789762907 -> 1789762908 00010000:00010000:0.0:1789762909.403534:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000891aaaa3/0x416193e012b05383 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 9 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7d6 expref: 44 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.403543:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e502f589/0x416193e012b05367 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 8 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7ba expref: 43 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.403572:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c2cc60cf/0x416193e012b0534b lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 7 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd797 expref: 42 pid: 10916 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.403580:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000016074eb0/0x416193e012b05352 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 6 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7a5 expref: 41 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.403588:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05344 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 5 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd790 expref: 40 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.403598:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008cdb5dbf/0x416193e012b0537c lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7cf expref: 39 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.403610:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000be714456/0x416193e012b0533d lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [1048576->3145727] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd789 expref: 38 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.403637:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762909.403671:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86f01c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:5592:x1876702015107328:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 229us (345us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762909.403679:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 31 00000100:00100000:3.0:1789762909.452864:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762909.452941:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015107584 00000100:00100000:0.0:1789762909.452950:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015107584 00000100:00100000:0.0:1789762909.453010:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 339 00000100:00080000:3.0:1789762909.453012:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762909.453022:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:15399:x1876702015107584:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762909.453043:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762909.453060:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b053a6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7f2 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.453171:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b053ad lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.453184:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b053ad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.453199:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b053ad lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762909.453207:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 16777216 00010000:00010000:0.0:1789762909.453397:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b053a6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7f2 expref: 12 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.453412:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b053ad lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7f2 expref: 11 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.453434:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff981ec2179800) 00010000:00010000:0.0:1789762909.453437:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762909.453439:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfce00, rc 0) 00000100:00100000:0.0:1789762909.453486:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15399:x1876702015107584:12345-192.168.203.48@tcp:101:dd.0 Request processed in 468us (633us total) trans 4294967589 rc 0/0 00000100:00100000:0.0:1789762909.453492:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 339 00000100:00100000:2.0:1789762909.454692:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.454751:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015107840 00000100:00100000:1.0:1789762909.454758:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015107840 00000100:00100000:1.0:1789762909.454798:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 340 00000100:00100000:1.0:1789762909.454809:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15401:x1876702015107840:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762909.454822:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762909.454834:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b053bb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd7f9 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.454884:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b053c2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.454894:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b053c2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.454904:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b053c2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762909.454913:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 16777216 00010000:00010000:1.0:1789762909.454995:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b053bb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7f9 expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.455016:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b053c2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7f9 expref: 13 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.455024:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff981ec2179800) 00010000:00010000:1.0:1789762909.455028:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789762909.455030:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4da00, rc 0) 00000100:00100000:1.0:1789762909.455066:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:15401:x1876702015107840:12345-192.168.203.48@tcp:101:dd.0 Request processed in 260us (375us total) trans 4294967590 rc 0/0 00000100:00100000:1.0:1789762909.455074:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 340 00000100:00100000:2.0:1789762909.455854:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762909.455895:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015107968 00000100:00100000:0.0:1789762909.455900:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015107968 00000100:00100000:0.0:1789762909.455933:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 206 00000100:00100000:0.0:1789762909.455939:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:15399:x1876702015107968:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:0.0:1789762909.455949:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762909.455965:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b053d0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd800 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762909.455989:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x16:0x0] initial LVB size: 8388608, mtime: 0x6aad9d5b, atime: 0x0, ctime: 0x6aad9d5b, blocks: 0x1 00010000:00010000:0.0:1789762909.456007:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=432s v=(0 0) 00010000:00010000:0.0:1789762909.456012:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762909.456025:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b053d0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd800 expref: 38 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.456041:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b053d0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd800 expref: 38 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.456053:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfcc00, rc 0) 00000100:00100000:0.0:1789762909.456118:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:15399:x1876702015107968:12345-192.168.203.48@tcp:101:dd.0 Request processed in 176us (262us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762909.456134:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 206 00000100:00100000:2.0:1789762909.456371:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762909.456419:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015108096 00000100:00100000:0.0:1789762909.456426:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015108096 00000100:00100000:0.0:1789762909.456471:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 32 00000100:00100000:2.0:1789762909.456473:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762909.456492:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:2366:x1876702015108096:12345-192.168.203.48@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1789762909.456504:0:7595:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015108224 00010000:00010000:0.0:1789762909.456505:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1789762909.456512:0:7595:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015108224 00000100:00100000:2.0:1789762909.456517:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:0.0:1789762909.456527:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b053ad lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7f2 expref: 14 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.456559:0:5791:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff981ec2179800) 00010000:00010000:0.0:1789762909.456564:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762909.456572:0:8405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 33 00000100:00100000:2.0:1789762909.456586:0:8405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:2363:x1876702015108224:12345-192.168.203.48@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1789762909.456591:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015108480 00000100:00100000:3.0:1789762909.456598:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015108480 00010000:00010000:2.0:1789762909.456599:0:8405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1789762909.456603:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:2366:x1876702015108096:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 120us (229us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762909.456614:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 32 00010000:00010000:2.0:1789762909.456620:0:8405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b053c2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd7f9 expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.456638:0:8405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff981ec2179800) 00010000:00010000:2.0:1789762909.456643:0:8405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762909.456649:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 108 00000100:00100000:3.0:1789762909.456661:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15401:x1876702015108480:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:2.0:1789762909.456670:0:8405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702015108224:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 88us (198us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762909.456676:0:8405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 33 00000100:00100000:3.0:1789762909.456856:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15401:x1876702015108480:12345-192.168.203.48@tcp:35:dd.0 Request processed in 197us (339us total) trans 4294967591 rc 0/0 00000100:00100000:3.0:1789762909.456873:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 108 00000100:00100000:2.0:1789762909.460304:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762909.460344:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015108608 00000100:00100000:0.0:1789762909.460351:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015108608 00002000:00100000:0.0:1789762909.460383:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x16:0x0] (0->1048575) req@ffff981ea0fc7800 x1876702015108608/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:690/0 lens 488/0 e 0 to 0 dl 1789762920 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762909.460394:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ea0fc7800 with x1876702015108608 ext(0->1048575) 00002000:00010000:0.0:1789762909.460398:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ea0fc7800 00000100:00100000:0.0:1789762909.460414:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409904398225 time 409904467558 state 0x0 ref 1 00000100:00100000:0.0:1789762909.460422:0:6654:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 9: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 409904398225 time 409904467558 state 0x0 ref 1 00000100:00100000:0.0:1789762909.460437:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 3 deadline 432764443504 time 432764343504 state 0x1 ref 1 00000100:00100000:0.0:1789762909.460476:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 432764443504 time 432764382211 state 0x0 ref 1 00000100:00100000:0.0:1789762909.460489:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702015108608:12345-192.168.203.48@tcp:3:dd.0 00000100:00100000:0.0:1789762909.464725:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702015108608:12345-192.168.203.48@tcp:3:dd.0 Request processed in 4233us (4414us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762909.464737:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1156 00000100:00100000:0.0:1789762909.464745:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 432764443504 time 432764382211 state 0x2 ref 0 00002000:00100000:0.0:1789762909.464754:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x16:0x0] (0->1048575) req@ffff981ea0fc7800 x1876702015108608/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:690/0 lens 488/440 e 0 to 0 dl 1789762920 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762909.464762:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ea0fc7800 with x1876702015108608 ext(0->1048575) 00002000:00010000:0.0:1789762909.464766:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ea0fc7800 00000100:00100000:2.0:1789762909.503194:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762909.503254:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015108864 00000100:00100000:0.0:1789762909.503266:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015108864 00000100:00100000:0.0:1789762909.503329:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 109 00000100:00100000:0.0:1789762909.503349:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15407:x1876702015108864:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:0.0:1789762909.503559:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:15407:x1876702015108864:12345-192.168.203.48@tcp:35:dd.0 Request processed in 210us (360us total) trans 4294967592 rc 0/0 00000100:00100000:0.0:1789762909.503567:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 109 00000100:00100000:2.0:1789762909.516141:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.516181:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015108992 00000100:00100000:1.0:1789762909.516191:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015108992 00000100:00100000:1.0:1789762909.516238:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 341 00000100:00100000:1.0:1789762909.516245:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:15415:x1876702015108992:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:1.0:1789762909.516266:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762909.516284:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b053d7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd807 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.516362:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c200/0x416193e012b053de lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.516378:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c200/0x416193e012b053de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.516391:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c200/0x416193e012b053de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762909.516423:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00000004:00200000:1.0:1789762909.516426:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762909.516438:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c200/0x416193e012b053de lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.516458:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b053d7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd807 expref: 15 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.516475:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c200/0x416193e012b053de lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd807 expref: 14 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.516484:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:1.0:1789762909.516488:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762909.516492:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4c200, rc 0) 00000100:00100000:1.0:1789762909.516557:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15415:x1876702015108992:12345-192.168.203.48@tcp:101:rm.0 Request processed in 299us (412us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762909.516566:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 341 00000100:00100000:2.0:1789762909.517588:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762909.517625:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015109248 00000100:00100000:0.0:1789762909.517631:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015109248 00000100:00100000:0.0:1789762909.517671:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 342 00000100:00100000:0.0:1789762909.517682:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15415:x1876702015109248:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:0.0:1789762909.517703:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762909.517712:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b053e5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd80e expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762909.517753:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00010000:00010000:0.0:1789762909.517836:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b053e5 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd80e expref: 16 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.517846:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:0.0:1789762909.517848:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789762909.517851:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b053e5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd80e expref: 16 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.517859:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfce00, rc 0) 00000100:00100000:0.0:1789762909.517888:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15415:x1876702015109248:12345-192.168.203.48@tcp:101:rm.0 Request processed in 210us (301us total) trans 4294967593 rc 301/301 00000100:00100000:0.0:1789762909.517895:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 342 00000100:00100000:2.0:1789762909.519320:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762909.519359:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015109376 00000100:00100000:3.0:1789762909.519365:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015109376 00000100:00100000:3.0:1789762909.519394:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 110 00000100:00100000:3.0:1789762909.519411:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15415:x1876702015109376:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:3.0:1789762909.521266:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15415:x1876702015109376:12345-192.168.203.48@tcp:37:rm.0 Request processed in 1863us (1944us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762909.521274:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 110 00000100:00100000:2.0:1789762909.522522:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762909.522566:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015109632 00000100:00100000:0.0:1789762909.522571:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015109632 00000100:00100000:0.0:1789762909.522596:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 111 00000100:00100000:0.0:1789762909.522602:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15415:x1876702015109632:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:0.0:1789762909.522691:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:15415:x1876702015109632:12345-192.168.203.48@tcp:35:rm.0 Request processed in 89us (170us total) trans 4294967594 rc 0/0 00000100:00100000:0.0:1789762909.522697:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 111 00000100:00100000:2.0:1789762909.523371:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.523409:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015109888 00000100:00100000:1.0:1789762909.523414:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015109888 00000100:00100000:1.0:1789762909.523441:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 343 00000100:00100000:1.0:1789762909.523448:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:15415:x1876702015109888:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:1.0:1789762909.523457:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762909.523465:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b053f3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd815 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.523508:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c000/0x416193e012b053fa lrc: 3/1,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.523524:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c000/0x416193e012b053fa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.523533:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c000/0x416193e012b053fa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762909.523543:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00010000:00010000:1.0:1789762909.523605:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b053f3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd815 expref: 19 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.523616:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c000/0x416193e012b053fa lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd815 expref: 18 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.523622:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:1.0:1789762909.523624:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1789762909.523626:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4c000, rc 0) 00000100:00100000:1.0:1789762909.523654:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdff100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15415:x1876702015109888:12345-192.168.203.48@tcp:101:rm.0 Request processed in 207us (284us total) trans 4294967595 rc 0/0 00000100:00100000:1.0:1789762909.523659:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 343 00000100:00100000:2.0:1789762909.524344:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.524374:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015110016 00000100:00100000:1.0:1789762909.524379:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015110016 00000100:00100000:1.0:1789762909.524404:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 344 00000100:00100000:1.0:1789762909.524411:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15415:x1876702015110016:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:1.0:1789762909.524441:0:6611:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7719616512 avail 7715422208 00000100:00100000:1.0:1789762909.524471:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15415:x1876702015110016:12345-192.168.203.48@tcp:41:rm.0 Request processed in 62us (128us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762909.524478:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 344 00000100:00100000:2.0:1789762909.525115:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762909.525143:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015110272 00000100:00100000:0.0:1789762909.525148:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015110272 00000100:00100000:0.0:1789762909.525172:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 345 00000100:00100000:0.0:1789762909.525184:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15415:x1876702015110272:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:0.0:1789762909.525204:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1789762909.525215:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b05328 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x39:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd77b expref: 18 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.525226:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x39:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762909.525234:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c200/0x416193e012b053de lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd807 expref: 17 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.525241:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762909.525243:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762909.525252:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b05408 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.525259:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b05408 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.525264:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b05408 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.525273:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b0540f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.525278:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b0540f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.525282:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b0540f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.525440:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05416 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x39:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.525450:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05416 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x39:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.525456:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05416 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x39:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789762909.526286:0:5804:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].14, 64 off9088 00000004:00080000:0.0:1789762909.526294:0:5804:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 432830206098ns, next at 433830204692ns, rc = 0 00000040:00080000:0.0:1789762909.526324:0:5804:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].22, 64 off9600 00000004:00080000:0.0:1789762909.526329:0:5804:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 432830241392ns, next at 433830240397ns, rc = 0 00010000:00010000:0.0:1789762909.526366:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b0541d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x39:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526373:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b0541d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x39:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526379:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b0541d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x39:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526385:0:5804:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b0541d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x39:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526390:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b0541d lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x39:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526394:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b0541d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x39:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526408:0:5804:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789762909.526416:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05416 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x39:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526428:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05416 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x39:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526433:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05416 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x39:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526437:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762909.526440:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05416 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x39:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526445:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05416 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x39:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526451:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x39:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762909.526454:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762909.526456:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05416 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x39:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.526460:0:5793:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762909.526461:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05416 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x39:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.526468:0:5793:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b0541d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x39:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526476:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b05408 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.526477:0:5793:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762909.526485:0:5793:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b0541d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x39:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526486:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b05408 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526503:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b05408 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.526507:0:5793:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b0541d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x39:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526508:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762909.526517:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b05408 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526536:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b05408 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.526540:0:5793:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x39:0x0].0x0 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762909.526556:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:1.0:1789762909.526557:0:5793:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762909.526563:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762909.526565:0:5793:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b0541d lrc: 1/0,0 mode: --/PW res: [0x200000401:0x39:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526566:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b05408 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.526582:0:5793:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b0541d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x39:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.526583:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd400/0x416193e012b05408 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762909.526605:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b089a897 reply state 0000000069206f21 transno 4294967596 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b0540f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.526608:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762909.526611:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762909.526613:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789762909.526614:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762909.526649:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15415:x1876702015110272:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1465us (1533us total) trans 4294967596 rc 0/0 00000100:00100000:0.0:1789762909.526655:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 345 00010000:00010000:3.0:1789762909.527110:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b0540f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.527121:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b0540f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.527126:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b0540f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.527131:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfde00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762909.527134:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b0540f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.527139:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b0540f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.527148:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x992ad74f (ffff981ea0aa3d00) 00010000:00010000:3.0:1789762909.527151:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762909.527155:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b0540f lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.527162:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b0540f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762909.527654:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.527686:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015110528 00000100:00100000:1.0:1789762909.527691:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015110528 00000100:00100000:1.0:1789762909.527717:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 346 00000100:00100000:1.0:1789762909.527723:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:15415:x1876702015110528:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:1.0:1789762909.527744:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762909.527755:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c800/0x416193e012b052f0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd766 expref: 17 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.527765:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff981ec2179800) 00010000:00010000:1.0:1789762909.527770:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762909.527778:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05424 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.527784:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05424 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.527789:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05424 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.527797:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b0542b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.527802:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b0542b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.527807:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b0542b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.527841:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b05432 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.527846:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b05432 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.527851:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b05432 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789762909.528109:0:5806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].23, 64 off9664 00000040:00080000:1.0:1789762909.528122:0:5806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].15, 64 off9152 00010000:00010000:1.0:1789762909.528154:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05439 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528161:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05439 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528167:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05439 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528172:0:5806:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05439 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528176:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05439 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528179:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05439 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528193:0:5806:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762909.528200:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b05432 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528205:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b05432 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528209:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b05432 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.528209:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762909.528216:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05439 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528218:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762909.528224:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b05432 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528237:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b05432 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.528244:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4da00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762909.528247:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff981ec2179d00) 00010000:00010000:1.0:1789762909.528250:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762909.528253:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b05432 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528258:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b05432 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.528258:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05439 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.528281:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05439 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528284:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05424 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528293:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05424 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.528294:0:8466:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff981ec2179d00) 00010000:00010000:3.0:1789762909.528301:0:8466:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789762909.528304:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05424 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.528305:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05439 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528316:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4c400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762909.528318:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4da00/0x416193e012b05439 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528320:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05424 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528330:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05424 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.528339:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762909.528340:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:3.0:1789762909.528342:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762909.528344:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762909.528345:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762909.528347:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05424 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528352:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05424 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762909.528359:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000076bf3817 reply state 000000005477591f transno 4294967597 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b0542b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762909.528368:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762909.528400:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb17100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15415:x1876702015110528:12345-192.168.203.48@tcp:36:rm.0 Request processed in 677us (746us total) trans 4294967597 rc 0/0 00000100:00100000:1.0:1789762909.528405:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 346 00010000:00010000:1.0:1789762909.528948:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b0542b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528957:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b0542b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528963:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b0542b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528969:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4dc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762909.528972:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b0542b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528979:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b0542b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.528989:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x77289514 (ffff981ec2179400) 00010000:00010000:1.0:1789762909.528992:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762909.528995:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b0542b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.529003:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b0542b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762909.530321:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762909.530397:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015110784 00000100:00100000:3.0:1789762909.530405:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015110784 00000100:00100000:3.0:1789762909.530438:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 112 00000100:00100000:3.0:1789762909.530445:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7ad5850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:15415:x1876702015110784:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762909.530570:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7ad5850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15415:x1876702015110784:12345-192.168.203.48@tcp:35:rm.0 Request processed in 123us (248us total) trans 4294967598 rc 0/0 00000100:00100000:3.0:1789762909.530579:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 112 00000100:00100000:2.0:1789762909.531290:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.531321:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015111040 00000100:00100000:1.0:1789762909.531327:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015111040 00000100:00100000:1.0:1789762909.531354:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 347 00000100:00100000:1.0:1789762909.531361:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15415:x1876702015111040:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:1.0:1789762909.531382:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1789762909.531393:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c000/0x416193e012b053fa lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd815 expref: 18 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531406:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc400/0x416193e012b052d4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd75f expref: 17 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531415:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:1.0:1789762909.531424:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4cc00/0x416193e012b05272 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd74a expref: 16 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531430:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179f00) 00010000:00010000:1.0:1789762909.531434:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762909.531444:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05440 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531451:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05440 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531457:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05440 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531464:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c600/0x416193e012b05447 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531469:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c600/0x416193e012b05447 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531477:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c600/0x416193e012b05447 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531613:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4de00/0x416193e012b0544e lrc: 3/0,1 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531623:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4de00/0x416193e012b0544e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531632:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4de00/0x416193e012b0544e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762909.531973:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00010000:00010000:1.0:1789762909.531981:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4de00/0x416193e012b0544e lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531986:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4de00/0x416193e012b0544e lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531990:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4de00/0x416193e012b0544e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.531994:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4de00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762909.531996:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4de00/0x416193e012b0544e lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.532000:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4de00/0x416193e012b0544e lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.532006:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff981ec2179500) 00010000:00010000:1.0:1789762909.532008:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762909.532011:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4de00/0x416193e012b0544e lrc: 2/0,0 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.532015:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4de00/0x416193e012b0544e lrc: 0/0,0 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.532026:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05440 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.532033:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05440 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.532039:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05440 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.532044:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762909.532047:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05440 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.532054:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05440 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.532062:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:1.0:1789762909.532065:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762909.532068:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05440 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.532076:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d000/0x416193e012b05440 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762909.532108:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005c604cdc reply state 0000000035d68f84 transno 4294967599 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c600/0x416193e012b05447 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762909.532119:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762909.532154:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:15415:x1876702015111040:12345-192.168.203.48@tcp:36:rm.0 Request processed in 793us (865us total) trans 4294967599 rc 0/0 00000100:00100000:1.0:1789762909.532159:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 347 00010000:00010000:3.0:1789762909.532669:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c600/0x416193e012b05447 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.532677:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c600/0x416193e012b05447 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.532682:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c600/0x416193e012b05447 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.532686:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4c600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762909.532688:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c600/0x416193e012b05447 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.532692:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c600/0x416193e012b05447 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.532699:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179100) 00010000:00010000:3.0:1789762909.532701:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762909.532703:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c600/0x416193e012b05447 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.532707:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c600/0x416193e012b05447 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762909.844823:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762909.844918:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015111168 00000100:00100000:3.0:1789762909.844936:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015111168 00000100:00100000:3.0:1789762909.845023:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 348 00000100:00100000:3.0:1789762909.845037:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:15426:x1876702015111168:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:3.0:1789762909.845069:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762909.845131:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05455 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd81c expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.845215:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6400/0x416193e012b0545c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.845236:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6400/0x416193e012b0545c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.845252:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6400/0x416193e012b0545c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762909.845296:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762909.845299:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762909.845342:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6400/0x416193e012b0545c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.845375:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05455 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd81c expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.845396:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6400/0x416193e012b0545c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd81c expref: 17 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.845406:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:3.0:1789762909.845410:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762909.845414:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d865a6400, rc 0) 00000100:00100000:3.0:1789762909.845504:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb17b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:15426:x1876702015111168:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 471us (685us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762909.845512:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 348 00000100:00100000:2.0:1789762909.846952:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.847002:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015111424 00000100:00100000:1.0:1789762909.847013:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015111424 00000100:00100000:1.0:1789762909.847067:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 349 00000100:00100000:1.0:1789762909.847077:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:15426:x1876702015111424:12345-192.168.203.48@tcp:36:mkdir.0 00010000:00010000:1.0:1789762909.847158:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762909.847182:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6400/0x416193e012b0545c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd81c expref: 17 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847197:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:1.0:1789762909.847206:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762909.847265:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05463 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847275:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05463 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847282:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05463 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847291:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0546a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847297:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0546a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847316:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0546a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762909.847871:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00010000:00010000:1.0:1789762909.847880:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05463 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847886:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05463 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847891:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05463 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847895:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4dc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762909.847898:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05463 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847903:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05463 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847910:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:1.0:1789762909.847912:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762909.847916:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05463 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.847922:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05463 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762909.847938:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008e28fc92 reply state 0000000019defe6c transno 4294967600 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0546a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762909.847954:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762909.848038:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15426:x1876702015111424:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 960us (1085us total) trans 4294967600 rc 0/0 00000100:00100000:1.0:1789762909.848046:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 349 00010000:00010000:1.0:1789762909.848780:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0546a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.848789:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0546a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.848795:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0546a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.848799:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4c400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762909.848802:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0546a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.848810:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0546a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.848827:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179500) 00010000:00010000:1.0:1789762909.848830:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762909.848834:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0546a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.848839:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b0546a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762909.855369:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.855445:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015111552 00000100:00100000:1.0:1789762909.855451:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015111552 00000100:00100000:1.0:1789762909.855486:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 350 00000100:00100000:1.0:1789762909.855495:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:15427:x1876702015111552:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762909.855507:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762909.855520:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05471 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd823 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.855596:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05478 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.855604:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05478 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.855610:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05478 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762909.855625:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762909.855628:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762909.855640:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05478 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.855657:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b05471 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd823 expref: 19 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.855670:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05478 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd823 expref: 18 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.855676:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179500) 00010000:00010000:1.0:1789762909.855678:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762909.855681:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4dc00, rc 0) 00000100:00100000:1.0:1789762909.855711:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15427:x1876702015111552:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 219us (345us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762909.855718:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 350 00000100:00100000:0.0:1789762909.857054:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762909.857161:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015111808 00000100:00100000:3.0:1789762909.857172:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015111808 00000100:00100000:3.0:1789762909.857243:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 351 00000100:00100000:3.0:1789762909.857263:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bce300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15427:x1876702015111808:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762909.857281:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762909.857286:0:5804:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1789762909.857319:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4dc00/0x416193e012b05478 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd823 expref: 18 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857342:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179500) 00010000:00010000:3.0:1789762909.857358:0:5804:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762909.857374:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b0547f lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd82a expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857600:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b05486 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857616:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b05486 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857628:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b05486 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857649:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b0548d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857659:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b0548d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857670:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b0548d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857790:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05494 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857804:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05494 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857825:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05494 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762909.857839:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00010000:00010000:3.0:1789762909.857858:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05494 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857870:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05494 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857882:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05494 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857893:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762909.857900:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05494 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857913:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05494 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857928:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981e9272c500) 00010000:00010000:3.0:1789762909.857932:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762909.857938:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05494 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857949:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05494 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857976:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b05486 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.857988:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b05486 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858000:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b05486 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858011:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762909.858016:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b05486 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858029:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b05486 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858044:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762909.858048:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762909.858053:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b05486 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858064:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b05486 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858081:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b0548d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858092:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b0548d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858152:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b0548d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858162:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762909.858167:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b0548d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858182:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b0548d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858194:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272c900) 00010000:00010000:3.0:1789762909.858198:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762909.858202:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b0548d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858212:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b0548d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858248:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b0547f lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd82a expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858261:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762909.858265:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762909.858270:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b0547f lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd82a expref: 18 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.858298:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d865a6600, rc 0) 00000100:00100000:3.0:1789762909.858367:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bce300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:15427:x1876702015111808:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 1111us (1311us total) trans 0 rc 301/301 00000100:00100000:3.0:1789762909.858381:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 351 00000100:00100000:2.0:1789762909.859558:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.859613:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015111936 00000100:00100000:1.0:1789762909.859620:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015111936 00000100:00100000:1.0:1789762909.859653:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 352 00000100:00100000:1.0:1789762909.859662:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:15427:x1876702015111936:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762909.859672:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762909.859685:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b0549b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd831 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.859715:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c800/0x416193e012b054a2 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.859724:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c800/0x416193e012b054a2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.859730:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c800/0x416193e012b054a2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762909.859746:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762909.859749:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762909.859758:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c800/0x416193e012b054a2 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.859771:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b0549b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd831 expref: 19 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.859782:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c800/0x416193e012b054a2 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd831 expref: 18 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.859792:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:1.0:1789762909.859794:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762909.859797:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4c800, rc 0) 00000100:00100000:1.0:1789762909.859830:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15427:x1876702015111936:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 170us (290us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762909.859835:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 352 00000100:00100000:2.0:1789762909.861179:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.861211:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015112192 00000100:00100000:1.0:1789762909.861216:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015112192 00000100:00100000:1.0:1789762909.861245:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 353 00000100:00100000:1.0:1789762909.861252:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15427:x1876702015112192:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762909.861259:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762909.861269:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b054a9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd838 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861300:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054b0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861307:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054b0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861314:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054b0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861324:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054b7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861329:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861337:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762909.861395:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00010000:00010000:1.0:1789762909.861458:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054b0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861464:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054b0 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861469:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054b0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861473:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4ce00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762909.861477:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054b0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861482:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054b0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861489:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:1.0:1789762909.861491:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762909.861493:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054b0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861497:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054b0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861504:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054b7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861512:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054b7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861516:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054b7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861520:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762909.861522:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054b7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861526:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054b7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861531:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179f00) 00010000:00010000:1.0:1789762909.861533:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762909.861535:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054b7 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861538:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054b7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861577:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b054a9 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd838 expref: 20 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861584:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:1.0:1789762909.861585:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789762909.861588:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b054a9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd838 expref: 20 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.861593:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4c400, rc 0) 00000100:00100000:1.0:1789762909.861627:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:15427:x1876702015112192:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 377us (455us total) trans 4294967601 rc 301/301 00000100:00100000:1.0:1789762909.861633:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 353 00000100:00100000:2.0:1789762909.862937:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762909.863022:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015112448 00000100:00100000:3.0:1789762909.863031:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015112448 00000100:00100000:3.0:1789762909.863074:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 354 00000100:00100000:3.0:1789762909.863171:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:15427:x1876702015112448:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:3.0:1789762909.863228:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b054c5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.863240:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b054c5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.863248:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b054c5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762909.863482:0:5804:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c813c299 reply state 0000000056a0f57a transno 4294967602 ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b054c5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762909.863494:0:5804:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762909.863502:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00000100:00100000:3.0:1789762909.863638:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:15427:x1876702015112448:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 551us (698us total) trans 4294967602 rc 0/0 00000100:00100000:3.0:1789762909.863652:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 354 00010000:00010000:3.0:1789762909.864369:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b054c5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.864380:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b054c5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.864387:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b054c5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.864394:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762909.864398:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b054c5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.864405:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b054c5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.864423:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981e9272c500) 00010000:00010000:3.0:1789762909.864437:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00000100:00100000:2.0:1789762909.864438:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:3.0:1789762909.864448:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b054c5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762909.864486:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015112704 00000100:00100000:1.0:1789762909.864494:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015112704 00010000:00010000:3.0:1789762909.864503:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b054c5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762909.864563:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 113 00000100:00100000:1.0:1789762909.864575:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:15427:x1876702015112704:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:1.0:1789762909.864710:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:15427:x1876702015112704:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 135us (284us total) trans 4294967603 rc 0/0 00000100:00100000:1.0:1789762909.864719:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 113 00000100:00100000:2.0:1789762909.868928:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.868963:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015112832 00000100:00100000:1.0:1789762909.868969:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015112832 00000100:00100000:1.0:1789762909.869009:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 355 00000100:00100000:1.0:1789762909.869020:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:15429:x1876702015112832:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:1.0:1789762909.869037:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762909.869055:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b054cc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd83f expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869140:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054d3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869162:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054d3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869174:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054d3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869192:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054da lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869201:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869209:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869295:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b054e1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869304:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b054e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869312:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b054e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762909.869332:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00000004:00200000:1.0:1789762909.869336:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762909.869354:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b054e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869365:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054d3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869373:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054d3 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869380:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054d3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869386:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762909.869391:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054d3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869410:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054d3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869424:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:1.0:1789762909.869428:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762909.869433:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054d3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869438:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d800/0x416193e012b054d3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869449:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054da lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869454:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054da lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869458:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054da lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869463:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4ce00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762909.869466:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054da lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869470:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054da lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869485:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179f00) 00010000:00010000:1.0:1789762909.869487:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762909.869489:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054da lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869509:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4ce00/0x416193e012b054da lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869563:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c400/0x416193e012b054cc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd83f expref: 23 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869575:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b054e1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd83f expref: 22 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.869581:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981ec2179400) 00010000:00010000:1.0:1789762909.869583:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762909.869586:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfa4d600, rc 0) 00000100:00100000:1.0:1789762909.869628:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb17480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:15429:x1876702015112832:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 612us (702us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762909.869633:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 355 00000100:00100000:3.0:1789762909.871396:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.871444:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015113088 00000100:00100000:1.0:1789762909.871450:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015113088 00000100:00100000:1.0:1789762909.871494:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 356 00000100:00100000:1.0:1789762909.871513:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:15429:x1876702015113088:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:1.0:1789762909.871538:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762909.871566:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d600/0x416193e012b054e1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd83f expref: 22 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.871582:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981ec2179400) 00010000:00010000:1.0:1789762909.871589:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762909.871604:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b054e8 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.871611:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b054e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.871617:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b054e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762909.871776:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b3738d61 reply state 00000000f41dd1ea transno 4294967604 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b054e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762909.871784:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:1.0:1789762909.871788:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00000100:00100000:1.0:1789762909.871838:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851ba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15429:x1876702015113088:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 335us (459us total) trans 4294967604 rc 0/0 00000100:00100000:1.0:1789762909.871844:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 356 00010000:00010000:2.0:1789762909.872478:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b054e8 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.872485:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b054e8 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.872490:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b054e8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.872494:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762909.872497:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b054e8 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.872501:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b054e8 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.872510:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981ec2179800) 00010000:00010000:2.0:1789762909.872512:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762909.872515:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b054e8 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.872519:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4d200/0x416193e012b054e8 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762909.893634:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762909.893684:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015113216 00000100:00100000:0.0:1789762909.893693:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015113216 00000100:00100000:0.0:1789762909.893732:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 357 00000100:00100000:0.0:1789762909.893739:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:15435:x1876702015113216:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762909.893754:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762909.893765:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b054ef lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd846 expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893814:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b054f6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893823:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b054f6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893829:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b054f6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893838:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b054fd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893842:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b054fd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893846:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b054fd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893907:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05504 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893912:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893916:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762909.893939:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00000004:00200000:0.0:1789762909.893941:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762909.893951:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893967:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b054f6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893971:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b054f6 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893976:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b054f6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893980:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762909.893982:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b054f6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893987:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b054f6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.893994:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:0.0:1789762909.893996:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762909.893998:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b054f6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.894002:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b054f6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.894016:0:5804:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b054fd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.894020:0:5804:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b054fd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.894024:0:5804:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b054fd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.894032:0:5804:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762909.894034:0:5804:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b054fd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.894038:0:5804:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b054fd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.894043:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762909.894045:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762909.894046:0:5804:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b054fd lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.894050:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b054fd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.894068:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b054ef lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd846 expref: 24 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.894078:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05504 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd846 expref: 23 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762909.894083:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981ea0aa3e00) 00010000:00010000:0.0:1789762909.894121:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789762909.894124:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfce00, rc 0) 00000100:00100000:0.0:1789762909.894163:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15435:x1876702015113216:12345-192.168.203.48@tcp:101:dd.500 Request processed in 425us (633us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762909.894169:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 357 00000100:00100000:1.0:1789762909.895493:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762909.895603:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015113472 00000100:00100000:3.0:1789762909.895615:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015113472 00000100:00100000:3.0:1789762909.895683:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 358 00000100:00100000:3.0:1789762909.895702:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15435:x1876702015113472:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762909.895724:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762909.895745:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b0550b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd84d expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.895903:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05512 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.895925:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05512 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.895941:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05512 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.895964:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b05519 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.895985:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b05519 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.895997:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b05519 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762909.896852:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x18:0x0], index 0: 0 00010000:00010000:3.0:1789762909.897207:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05520 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.897228:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05520 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.897246:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05520 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.897321:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05512 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.897333:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05512 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.897344:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05512 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.897356:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762909.897380:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05512 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.897395:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05512 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.897424:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981ea0aa3e00) 00010000:00010000:3.0:1789762909.897430:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762909.897438:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05512 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.897451:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05512 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762909.897471:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006d369820 reply state 000000009006506f transno 4294967605 ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b05519 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762909.897487:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789762909.897540:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6a00/0x416193e012b0550b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd84d expref: 26 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.897607:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05520 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd84d expref: 25 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.897622:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762909.897627:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762909.897631:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d865a7000, rc 0) 00000100:00100000:3.0:1789762909.897707:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:15435:x1876702015113472:12345-192.168.203.48@tcp:101:dd.500 Request processed in 2010us (2211us total) trans 4294967605 rc 0/0 00000100:00100000:3.0:1789762909.897721:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 358 00010000:00010000:3.0:1789762909.898517:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b05519 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.898527:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b05519 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.898535:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b05519 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.898566:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762909.898571:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b05519 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.898600:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b05519 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.898618:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x4009f790 (ffff981e9272c500) 00010000:00010000:3.0:1789762909.898624:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762909.898630:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b05519 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.898642:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6e00/0x416193e012b05519 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762909.901654:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762909.901761:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015113600 00000100:00100000:3.0:1789762909.901770:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015113600 00000100:00100000:3.0:1789762909.901821:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 432663286587 time 432663427540 state 0x0 ref 1 00000100:00100000:3.0:1789762909.901838:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 432863427540 time 432663427540 state 0x1 ref 1 00000100:00100000:3.0:1789762909.901890:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 432863427540 time 433205778936 state 0x0 ref 1 00000100:00100000:3.0:1789762909.901904:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2366:x1876702015113600:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762909.901949:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762909.901957:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093a06fa6/0x416193e012b0552e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.901973:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=433s v=(0 0) 00010000:00010000:3.0:1789762909.901996:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762909.902003:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000093a06fa6/0x416193e012b0552e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762909.902020:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000093a06fa6/0x416193e012b0552e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762909.936901:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968144 is committed 00002000:00010000:2.0:1789762909.936926:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:2.0:1789762909.936931:0:12582:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 0 -> 1789762910 00002000:00010000:2.0:1789762909.936934:0:12582:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 0 -> 1789762910 00000001:00080000:1.0:1789762909.936938:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968148 is committed 00002000:00010000:2.0:1789762909.936945:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:2.0:1789762909.936957:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000093a06fa6/0x416193e012b0552e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762909.936966:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968149 is committed 00010000:00010000:2.0:1789762909.936969:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000093a06fa6/0x416193e012b0552e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.936978:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000093a06fa6/0x416193e012b0552e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.936984:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762909.936988:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000093a06fa6/0x416193e012b0552e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.936995:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000093a06fa6/0x416193e012b0552e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.937011:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000093a06fa6/0x416193e012b0552e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.937017:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000093a06fa6/0x416193e012b0552e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762909.937133:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015113600:12345-192.168.203.48@tcp:4:dd.500 Request processed in 35229us (35480us total) trans 4294968149 rc 0/0 00000100:00100000:2.0:1789762909.937143:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1157 00000100:00100000:2.0:1789762909.937154:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 432863427540 time 433205778936 state 0x2 ref 0 00000100:00100000:2.0:1789762909.950744:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762909.950787:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015113728 00000100:00100000:1.0:1789762909.950796:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015113728 00000100:00100000:1.0:1789762909.950836:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 432863427540 time 433205778936 state 0x0 ref 1 00000100:00100000:1.0:1789762909.950847:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 433405778936 time 433205778936 state 0x1 ref 1 00000100:00100000:1.0:1789762909.950875:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433405778936 time 433254782043 state 0x0 ref 1 00000100:00100000:1.0:1789762909.950888:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015113728:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762909.950930:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 1048576, mtime: 0x6aad9d5e, atime: 0x0, ctime: 0x6aad9d5e, blocks: 0x803 00010000:00010000:1.0:1789762909.950937:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b05535 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.950948:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=433s v=(0 0) 00010000:00010000:1.0:1789762909.950958:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762909.950963:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b05535 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762909.950973:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b05535 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762909.964040:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968150 is committed 00002000:00010000:2.0:1789762909.964130:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:2.0:1789762909.964135:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:2.0:1789762909.964146:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b05535 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.964156:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b05535 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.964162:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b05535 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.964167:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4c400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762909.964171:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b05535 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.964177:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b05535 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.964191:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b05535 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762909.964196:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000010af82ff/0x416193e012b05535 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762909.964262:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015113728:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13374us (13518us total) trans 4294968150 rc 0/0 00000100:00100000:2.0:1789762909.964278:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1158 00000100:00100000:2.0:1789762909.964288:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433405778936 time 433254782043 state 0x2 ref 0 00000100:00100000:1.0:1789762909.967001:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762909.967044:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015113856 00000100:00100000:3.0:1789762909.967052:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015113856 00000100:00100000:3.0:1789762909.967114:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433405778936 time 433254782043 state 0x0 ref 1 00000100:00100000:3.0:1789762909.967126:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433454782043 time 433254782043 state 0x1 ref 1 00010000:00010000:2.0:1789762910.135272:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762910.150969:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433454782043 time 433454867675 state 0x0 ref 1 00000100:00100000:0.0:1789762910.151005:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015113856:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762910.151062:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 2097152, mtime: 0x6aad9d5e, atime: 0x0, ctime: 0x6aad9d5e, blocks: 0x1007 00010000:00010000:0.0:1789762910.151069:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b0553c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.151082:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=433s v=(0 0) 00010000:00010000:0.0:1789762910.151102:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762910.151108:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b0553c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762910.151118:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b0553c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762910.166462:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968151 is committed 00002000:00010000:2.0:1789762910.166551:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:2.0:1789762910.166557:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:2.0:1789762910.166581:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b0553c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.166595:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b0553c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.166605:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b0553c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.166613:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762910.166616:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b0553c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.166624:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b0553c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.166639:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b0553c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.166645:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b0553c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762910.166723:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015113856:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15721us (199721us total) trans 4294968151 rc 0/0 00000100:00100000:2.0:1789762910.166732:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1159 00000100:00100000:2.0:1789762910.166741:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433454782043 time 433454867675 state 0x2 ref 0 00000100:00100000:2.0:1789762910.170449:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762910.170554:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015113984 00000100:00100000:1.0:1789762910.170575:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015113984 00000100:00100000:1.0:1789762910.170616:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433454782043 time 433454867675 state 0x0 ref 1 00000100:00100000:1.0:1789762910.170627:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433654867675 time 433454867675 state 0x1 ref 1 00000100:00100000:0.0:1789762910.306307:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762910.306360:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762910.306387:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762910.306412:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015114112 00000100:00100000:2.0:1789762910.306423:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015114112 00000100:00100000:3.0:1789762910.306428:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015114240 00000100:00100000:3.0:1789762910.306435:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015114240 00000100:00100000:1.0:1789762910.306463:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015114368 00000100:00100000:1.0:1789762910.306470:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015114368 00000100:00100000:2.0:1789762910.306472:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 342 00000100:00100000:2.0:1789762910.306485:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850da050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015114112:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:0.0:1789762910.306488:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762910.306495:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762910.306506:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 170 00000100:00100000:1.0:1789762910.306523:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 343 00000100:00100000:3.0:1789762910.306525:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2364:x1876702015114240:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762910.306536:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015114368:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762910.306564:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850da050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015114112:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 81us (202us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762910.306572:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 342 00000100:00100000:3.0:1789762910.306595:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2364:x1876702015114240:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 77us (296us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762910.306606:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015114368:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 72us (214us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762910.306616:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 343 00000100:00100000:3.0:1789762910.306621:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 170 00000100:00100000:2.0:1789762910.327224:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789762910.327240:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4bcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022188928:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762910.327276:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762910.327315:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022188928 00000100:00100000:3.0:1789762910.327321:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022188928 00000100:00080000:0.0:1789762910.327358:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762910.327375:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 143 00000100:00100000:3.0:1789762910.327385:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022188928:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762910.327402:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526188544 pending 0 free 3859808256 avail 3837788160 00000100:00100000:3.0:1789762910.327450:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb14700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022188928:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (171us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762910.327459:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 143 00000100:00100000:2.0:1789762910.327490:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4bcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022188928:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762910.351048:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433654867675 time 433654947322 state 0x0 ref 1 00000100:00100000:2.0:1789762910.351066:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb15880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015113984:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762910.351135:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 3145728, mtime: 0x6aad9d5e, atime: 0x0, ctime: 0x6aad9d5e, blocks: 0x1807 00010000:00010000:2.0:1789762910.351142:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05543 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.351167:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=433s v=(0 0) 00010000:00010000:2.0:1789762910.351170:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762910.351175:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05543 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.351187:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05543 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762910.362926:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968152 is committed 00002000:00010000:2.0:1789762910.362977:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:2.0:1789762910.362983:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:2.0:1789762910.362994:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05543 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.363008:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05543 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.363019:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05543 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.363028:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762910.363033:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05543 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.363044:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05543 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.363076:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05543 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.363103:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05543 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762910.363170:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb15880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015113984:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12105us (192721us total) trans 4294968152 rc 0/0 00000100:00100000:2.0:1789762910.363178:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1160 00000100:00100000:2.0:1789762910.363186:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433654867675 time 433654947322 state 0x2 ref 0 00000100:00100000:0.0:1789762910.365390:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762910.365431:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015114496 00000100:00100000:2.0:1789762910.365440:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015114496 00000100:00100000:2.0:1789762910.365478:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433654867675 time 433654947322 state 0x0 ref 1 00000100:00100000:2.0:1789762910.365489:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433854947322 time 433654947322 state 0x1 ref 1 00000100:00100000:2.0:1789762910.551141:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433854947322 time 433855035497 state 0x0 ref 1 00000100:00100000:2.0:1789762910.551166:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015114496:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762910.551215:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 4194304, mtime: 0x6aad9d5e, atime: 0x0, ctime: 0x6aad9d5e, blocks: 0x2007 00010000:00010000:2.0:1789762910.551223:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b0554a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.551250:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=433s v=(0 0) 00010000:00010000:2.0:1789762910.551254:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762910.551259:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b0554a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.551270:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b0554a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762910.564850:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968153 is committed 00002000:00010000:2.0:1789762910.564917:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 4194304 -> 5242880 00002000:00010000:2.0:1789762910.564922:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:2.0:1789762910.564933:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b0554a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.564943:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b0554a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.564950:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b0554a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.564955:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762910.564958:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b0554a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.564965:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b0554a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.564991:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b0554a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.564997:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b0554a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762910.565071:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015114496:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13912us (199678us total) trans 4294968153 rc 0/0 00000100:00100000:2.0:1789762910.565080:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1161 00000100:00100000:2.0:1789762910.565126:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433854947322 time 433855035497 state 0x2 ref 0 00000100:00100000:3.0:1789762910.567607:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762910.567652:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015114624 00000100:00100000:1.0:1789762910.567661:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015114624 00000100:00100000:1.0:1789762910.567702:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 433854947322 time 433855035497 state 0x0 ref 1 00000100:00100000:1.0:1789762910.567713:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434055035497 time 433855035497 state 0x1 ref 1 00000100:00100000:2.0:1789762910.751244:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434055035497 time 434055140638 state 0x0 ref 1 00000100:00100000:2.0:1789762910.751268:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015114624:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762910.751319:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 5242880, mtime: 0x6aad9d5e, atime: 0x0, ctime: 0x6aad9d5e, blocks: 0x2807 00010000:00010000:2.0:1789762910.751327:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05551 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.751341:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=434s v=(0 0) 00010000:00010000:2.0:1789762910.751344:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762910.751349:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05551 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.751362:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05551 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762910.765265:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968154 is committed 00002000:00010000:1.0:1789762910.765340:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:1.0:1789762910.765346:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:1.0:1789762910.765360:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05551 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.765375:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05551 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.765385:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05551 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.765393:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762910.765398:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05551 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.765408:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05551 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.765427:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05551 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762910.765441:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2a92c03/0x416193e012b05551 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762910.765527:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015114624:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14262us (197916us total) trans 4294968154 rc 0/0 00000100:00100000:1.0:1789762910.765554:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1162 00000100:00100000:1.0:1789762910.765568:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434055035497 time 434055140638 state 0x2 ref 0 00000100:00100000:0.0:1789762910.767632:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762910.767679:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015114752 00000100:00100000:1.0:1789762910.767687:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015114752 00000100:00100000:1.0:1789762910.767727:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434055035497 time 434055140638 state 0x0 ref 1 00000100:00100000:1.0:1789762910.767738:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434255140638 time 434055140638 state 0x1 ref 1 00000004:00080000:0.0:1789762910.839260:0:7726:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 434143170557ns, next at 435143167655ns, rc = 0 00000100:00100000:2.0:1789762910.839263:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4bcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022189184:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762910.839287:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc437480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022189312:0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.839301:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc435f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022189440:0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.839319:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc437100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022189568:0@lo:400:kworker.0 00000100:00100000:2.0:1789762910.839337:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762910.839346:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762910.839365:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022189184 00000100:00100000:0.0:1789762910.839373:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022189184 00000100:00100000:1.0:1789762910.839376:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762910.839393:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc436d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022189056:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.839404:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc435180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022189824:0@lo:400:kworker.0 00000100:00080000:1.0:1789762910.839426:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762910.839430:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762910.839449:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762910.839459:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022189312 00000100:00100000:0.0:1789762910.839464:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:1.0:1789762910.839467:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022189312 00000100:00100000:0.0:1789762910.839477:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022189184:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762910.839477:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc434e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022189696:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.839502:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022189568 00000100:00100000:1.0:1789762910.839508:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:3.0:1789762910.839510:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022189568 00000100:00100000:2.0:1789762910.839514:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00000024:0.0:1789762910.839515:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 98762752 pending 27262976 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762910.839521:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022189312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762910.839567:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022189056 00000100:00100000:2.0:1789762910.839572:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022189056 00000100:00100000:3.0:1789762910.839573:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022189824 00000100:00100000:3.0:1789762910.839579:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022189824 00000100:00100000:0.0:1789762910.839587:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022189184:12345-0@lo:13:osp-pre-1-0.0 Request processed in 114us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762910.839595:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00100000:1.0:1789762910.839601:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022189312:12345-0@lo:400:kworker.0 Request processed in 82us (257us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762910.839614:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:2.0:1789762910.839614:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 343 00000100:00100000:3.0:1789762910.839621:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:2.0:1789762910.839628:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850db050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022189056:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.839637:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022189568:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.839638:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022189696 00000100:00100000:0.0:1789762910.839639:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762910.839644:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022189696 00000100:00100000:1.0:1789762910.839682:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022189440 00000100:00100000:2.0:1789762910.839684:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850db050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022189056:12345-0@lo:400:kworker.0 Request processed in 57us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762910.839685:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022189440 00000100:00100000:2.0:1789762910.839696:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 343 00000100:00100000:0.0:1789762910.839703:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 345 00000100:00100000:3.0:1789762910.839711:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022189568:12345-0@lo:400:kworker.0 Request processed in 78us (334us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762910.839717:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022189696:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762910.839722:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:3.0:1789762910.839726:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:1.0:1789762910.839737:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022189440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.839754:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:2.0:1789762910.839763:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4bcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022189184:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762910.839770:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022189824:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.839771:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022189696:12345-0@lo:400:kworker.0 Request processed in 58us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762910.839780:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc436d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022189056:0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.839787:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 345 00000100:00100000:1.0:1789762910.839787:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022189440:12345-0@lo:400:kworker.0 Request processed in 52us (147us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762910.839797:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00100000:2.0:1789762910.839815:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc434e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022189696:0@lo:400:kworker.0 00000100:00080000:1.0:1789762910.839825:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762910.839833:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc435f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022189440:0@lo:400:kworker.0 00000100:00100000:3.0:1789762910.839840:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022189824:12345-0@lo:400:kworker.0 Request processed in 76us (392us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762910.839849:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00100000:2.0:1789762910.839880:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc437100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022189568:0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.839888:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc435180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022189824:0@lo:400:kworker.0 00000100:00100000:0.0:1789762910.839897:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc437480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022189312:0@lo:400:kworker.0 00000100:00100000:2.0:1789762910.951354:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434255140638 time 434255249232 state 0x0 ref 1 00000100:00100000:2.0:1789762910.951379:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015114752:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762910.951432:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 6291456, mtime: 0x6aad9d5e, atime: 0x0, ctime: 0x6aad9d5e, blocks: 0x3007 00010000:00010000:2.0:1789762910.951452:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05558 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.951465:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=434s v=(0 0) 00010000:00010000:2.0:1789762910.951469:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762910.951473:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05558 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762910.951484:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05558 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762910.965514:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968155 is committed 00002000:00010000:3.0:1789762910.965596:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:3.0:1789762910.965603:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:3.0:1789762910.965617:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05558 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.965631:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05558 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.965641:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05558 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.965649:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762910.965654:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05558 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.965665:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05558 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.965685:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05558 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762910.965693:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05558 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762910.965776:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015114752:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14402us (198141us total) trans 4294968155 rc 0/0 00000100:00100000:3.0:1789762910.965787:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1163 00000100:00100000:3.0:1789762910.965800:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434255140638 time 434255249232 state 0x2 ref 0 00000100:00100000:3.0:1789762910.967904:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762910.967943:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015114880 00000100:00100000:2.0:1789762910.967951:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015114880 00000100:00100000:2.0:1789762910.967990:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434255140638 time 434255249232 state 0x0 ref 1 00000100:00100000:2.0:1789762910.968001:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434455249232 time 434255249232 state 0x1 ref 1 00000100:00100000:2.0:1789762911.151487:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434455249232 time 434455382315 state 0x0 ref 1 00000100:00100000:2.0:1789762911.151548:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015114880:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:1.0:1789762911.151577:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762911.151603:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 7340032, mtime: 0x6aad9d5e, atime: 0x0, ctime: 0x6aad9d5e, blocks: 0x3807 00010000:00010000:2.0:1789762911.151612:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0555f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.151625:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=434s v=(0 0) 00010000:00010000:2.0:1789762911.151642:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762911.151647:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0555f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.151659:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0555f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762911.159178:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:2.0:1789762911.165834:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:2.0:1789762911.165841:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762910 -> 1789762911 00002000:00010000:2.0:1789762911.165844:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762910 -> 1789762911 00002000:00010000:2.0:1789762911.165847:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:2.0:1789762911.165861:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0555f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.165876:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0555f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.165885:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0555f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.165891:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762911.165894:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0555f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.165901:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0555f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.165918:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0555f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.165931:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000092259245/0x416193e012b0555f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762911.166016:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702015114880:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14509us (198109us total) trans 4294968156 rc 0/0 00000100:00100000:2.0:1789762911.166025:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1164 00000100:00100000:2.0:1789762911.166035:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434455249232 time 434455382315 state 0x2 ref 0 00000001:00080000:2.0:1789762911.166068:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968156 is committed 00000100:00100000:2.0:1789762911.168282:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762911.168331:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015115008 00000100:00100000:1.0:1789762911.168339:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015115008 00000100:00100000:1.0:1789762911.168388:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434455249232 time 434455382315 state 0x0 ref 1 00000100:00100000:1.0:1789762911.168404:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434655382315 time 434455382315 state 0x1 ref 1 00000100:00100000:1.0:1789762911.351583:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434655382315 time 434655473875 state 0x0 ref 1 00000100:00100000:1.0:1789762911.351615:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015115008:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762911.351674:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 8388608, mtime: 0x6aad9d5f, atime: 0x0, ctime: 0x6aad9d5f, blocks: 0x4007 00010000:00010000:1.0:1789762911.351686:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8f0f74b/0x416193e012b05566 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.351703:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=434s v=(0 0) 00010000:00010000:1.0:1789762911.351708:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762911.351715:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a8f0f74b/0x416193e012b05566 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762911.351731:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a8f0f74b/0x416193e012b05566 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762911.365666:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968157 is committed 00002000:00010000:2.0:1789762911.365698:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:2.0:1789762911.365704:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:2.0:1789762911.365721:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a8f0f74b/0x416193e012b05566 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.365733:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a8f0f74b/0x416193e012b05566 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.365740:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a8f0f74b/0x416193e012b05566 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.365746:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4ca00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762911.365751:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8f0f74b/0x416193e012b05566 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.365773:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a8f0f74b/0x416193e012b05566 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.365794:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a8f0f74b/0x416193e012b05566 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.365799:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a8f0f74b/0x416193e012b05566 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762911.365916:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015115008:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14308us (197628us total) trans 4294968157 rc 0/0 00000100:00100000:2.0:1789762911.365928:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1165 00000100:00100000:2.0:1789762911.365942:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434655382315 time 434655473875 state 0x2 ref 0 00000100:00100000:0.0:1789762911.368407:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762911.368469:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015115136 00000100:00100000:1.0:1789762911.368489:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015115136 00000100:00100000:1.0:1789762911.368566:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434655382315 time 434655473875 state 0x0 ref 1 00000100:00100000:1.0:1789762911.368583:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434855473875 time 434655473875 state 0x1 ref 1 00000100:00100000:3.0:1789762911.551665:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434855473875 time 434855556895 state 0x0 ref 1 00000100:00100000:3.0:1789762911.551697:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015115136:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762911.551752:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 9437184, mtime: 0x6aad9d5f, atime: 0x0, ctime: 0x6aad9d5f, blocks: 0x4807 00010000:00010000:3.0:1789762911.551761:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b0556d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.551778:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=434s v=(0 0) 00010000:00010000:3.0:1789762911.551783:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762911.551790:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b0556d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.551804:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b0556d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762911.567287:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968158 is committed 00002000:00010000:3.0:1789762911.567339:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:3.0:1789762911.567343:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 18439 -> 20487 00010000:00010000:3.0:1789762911.567355:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b0556d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.567366:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b0556d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.567374:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b0556d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.567380:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762911.567383:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b0556d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.567389:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b0556d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.567404:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b0556d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762911.567409:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b0556d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762911.567491:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015115136:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15804us (199085us total) trans 4294968158 rc 0/0 00000100:00100000:3.0:1789762911.567504:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1166 00000100:00100000:3.0:1789762911.567518:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434855473875 time 434855556895 state 0x2 ref 0 00000100:00100000:0.0:1789762911.570167:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762911.570224:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015115264 00000100:00100000:2.0:1789762911.570238:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015115264 00000100:00100000:2.0:1789762911.570302:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 434855473875 time 434855556895 state 0x0 ref 1 00000100:00100000:2.0:1789762911.570317:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435055556895 time 434855556895 state 0x1 ref 1 00000100:00100000:2.0:1789762911.751754:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435055556895 time 435055644213 state 0x0 ref 1 00000100:00100000:2.0:1789762911.751790:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015115264:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762911.751876:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 10485760, mtime: 0x6aad9d5f, atime: 0x0, ctime: 0x6aad9d5f, blocks: 0x5007 00010000:00010000:2.0:1789762911.751901:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05574 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.751919:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=435s v=(0 0) 00010000:00010000:2.0:1789762911.751923:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762911.751940:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05574 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.751956:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05574 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762911.765996:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968159 is committed 00002000:00010000:2.0:1789762911.766016:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:2.0:1789762911.766022:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:2.0:1789762911.766037:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05574 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.766048:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05574 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.766056:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05574 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.766061:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762911.766066:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05574 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.766074:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05574 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.766124:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05574 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.766130:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05574 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762911.766220:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015115264:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14445us (196050us total) trans 4294968159 rc 0/0 00000100:00100000:2.0:1789762911.766230:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1167 00000100:00100000:2.0:1789762911.766239:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435055556895 time 435055644213 state 0x2 ref 0 00000100:00100000:0.0:1789762911.768626:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762911.768674:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015115392 00000100:00100000:2.0:1789762911.768684:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015115392 00000100:00100000:2.0:1789762911.768733:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435055556895 time 435055644213 state 0x0 ref 1 00000100:00100000:2.0:1789762911.768747:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435255644213 time 435055644213 state 0x1 ref 1 00000100:00100000:2.0:1789762911.951823:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435255644213 time 435255711427 state 0x0 ref 1 00000100:00100000:2.0:1789762911.951865:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015115392:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762911.951940:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 11534336, mtime: 0x6aad9d5f, atime: 0x0, ctime: 0x6aad9d5f, blocks: 0x5807 00010000:00010000:2.0:1789762911.951948:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0557b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.951991:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=435s v=(0 0) 00010000:00010000:2.0:1789762911.951998:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762911.952005:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0557b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.952024:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0557b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762911.965546:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968160 is committed 00002000:00010000:2.0:1789762911.965607:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 11534336 -> 12582912 00002000:00010000:2.0:1789762911.965613:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 22535 -> 24583 00010000:00010000:2.0:1789762911.965626:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0557b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.965637:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0557b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.965644:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0557b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.965650:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762911.965654:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0557b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.965662:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0557b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.965692:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0557b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762911.965698:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0557b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762911.965775:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015115392:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13928us (197146us total) trans 4294968160 rc 0/0 00000100:00100000:2.0:1789762911.965785:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1168 00000100:00100000:2.0:1789762911.965796:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435255644213 time 435255711427 state 0x2 ref 0 00000100:00100000:3.0:1789762911.967938:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762911.967992:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015115520 00000100:00100000:1.0:1789762911.968003:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015115520 00000100:00100000:1.0:1789762911.968058:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435255644213 time 435255711427 state 0x0 ref 1 00000100:00100000:1.0:1789762911.968072:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435455711427 time 435255711427 state 0x1 ref 1 00000100:00100000:3.0:1789762912.151941:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435455711427 time 435455824120 state 0x0 ref 1 00000100:00100000:3.0:1789762912.151982:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015115520:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762912.152056:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 12582912, mtime: 0x6aad9d5f, atime: 0x0, ctime: 0x6aad9d5f, blocks: 0x6007 00010000:00010000:3.0:1789762912.152068:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b05582 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.152125:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=435s v=(0 0) 00010000:00010000:3.0:1789762912.152131:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762912.152137:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b05582 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762912.152152:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b05582 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762912.165250:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968161 is committed 00002000:00010000:2.0:1789762912.165293:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 12582912 -> 13631488 00002000:00010000:2.0:1789762912.165299:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762911 -> 1789762912 00002000:00010000:2.0:1789762912.165303:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762911 -> 1789762912 00002000:00010000:2.0:1789762912.165306:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:2.0:1789762912.165324:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b05582 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.165341:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b05582 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.165353:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b05582 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.165361:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762912.165377:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b05582 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.165389:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b05582 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.165416:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b05582 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.165425:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003848790a/0x416193e012b05582 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762912.165560:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015115520:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13589us (197617us total) trans 4294968161 rc 0/0 00000100:00100000:2.0:1789762912.165577:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1169 00000100:00100000:2.0:1789762912.165594:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435455711427 time 435455824120 state 0x2 ref 0 00000100:00100000:0.0:1789762912.167978:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762912.168021:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015115648 00000100:00100000:2.0:1789762912.168033:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015115648 00000100:00100000:2.0:1789762912.168111:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435455711427 time 435455824120 state 0x0 ref 1 00000100:00100000:2.0:1789762912.168125:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435655824120 time 435455824120 state 0x1 ref 1 00010000:00010000:1.0:1789762912.183248:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762912.352014:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435655824120 time 435655901373 state 0x0 ref 1 00000100:00100000:2.0:1789762912.352051:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015115648:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762912.352184:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 13631488, mtime: 0x6aad9d60, atime: 0x0, ctime: 0x6aad9d60, blocks: 0x6807 00010000:00010000:2.0:1789762912.352197:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05589 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.352218:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=435s v=(0 0) 00010000:00010000:2.0:1789762912.352223:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762912.352229:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05589 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.352244:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05589 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762912.364445:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968162 is committed 00002000:00010000:2.0:1789762912.364476:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:2.0:1789762912.364483:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:2.0:1789762912.364500:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05589 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.364517:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05589 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.364545:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05589 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.364555:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762912.364561:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05589 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.364588:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05589 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.364614:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05589 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.364624:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05589 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762912.364735:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015115648:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12687us (196744us total) trans 4294968162 rc 0/0 00000100:00100000:2.0:1789762912.364749:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1170 00000100:00100000:2.0:1789762912.364764:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435655824120 time 435655901373 state 0x2 ref 0 00000100:00100000:3.0:1789762912.367225:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762912.367263:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015115776 00000100:00100000:2.0:1789762912.367273:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015115776 00000100:00100000:2.0:1789762912.367323:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435655824120 time 435655901373 state 0x0 ref 1 00000100:00100000:2.0:1789762912.367336:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435855901373 time 435655901373 state 0x1 ref 1 00000001:00080000:1.0:1789762912.397978:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967432 is committed 00000100:00100000:2.0:1789762912.552165:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435855901373 time 435856047963 state 0x0 ref 1 00000100:00100000:2.0:1789762912.552209:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015115776:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762912.552286:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 14680064, mtime: 0x6aad9d60, atime: 0x0, ctime: 0x6aad9d60, blocks: 0x7007 00010000:00010000:2.0:1789762912.552315:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05590 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.552335:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=435s v=(0 0) 00010000:00010000:2.0:1789762912.552342:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762912.552350:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05590 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.552367:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05590 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762912.565011:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968163 is committed 00002000:00010000:2.0:1789762912.565101:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:2.0:1789762912.565107:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:2.0:1789762912.565121:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05590 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.565135:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05590 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.565145:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05590 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.565154:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762912.565160:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05590 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.565171:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05590 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.565203:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05590 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.565212:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05590 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762912.565303:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015115776:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13109us (198077us total) trans 4294968163 rc 0/0 00000100:00100000:2.0:1789762912.565316:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1171 00000100:00100000:2.0:1789762912.565330:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435855901373 time 435856047963 state 0x2 ref 0 00000100:00100000:0.0:1789762912.567475:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762912.567540:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015115904 00000100:00100000:1.0:1789762912.567552:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015115904 00000100:00100000:1.0:1789762912.567614:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 435855901373 time 435856047963 state 0x0 ref 1 00000100:00100000:1.0:1789762912.567633:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436056047963 time 435856047963 state 0x1 ref 1 00000100:00100000:2.0:1789762912.752265:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436056047963 time 436056154523 state 0x0 ref 1 00000100:00100000:2.0:1789762912.752301:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015115904:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762912.752372:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 15728640, mtime: 0x6aad9d60, atime: 0x0, ctime: 0x6aad9d60, blocks: 0x7807 00010000:00010000:2.0:1789762912.752382:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05597 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.752400:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=436s v=(0 0) 00010000:00010000:2.0:1789762912.752404:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762912.752410:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05597 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.752426:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05597 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762912.768548:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968164 is committed 00002000:00010000:2.0:1789762912.768606:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:2.0:1789762912.768612:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:2.0:1789762912.768626:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05597 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.768638:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05597 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.768645:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05597 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.768650:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762912.768654:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05597 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.768661:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05597 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.768679:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05597 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.768685:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b05597 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762912.768796:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015115904:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16506us (201315us total) trans 4294968164 rc 0/0 00000100:00100000:2.0:1789762912.768813:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1172 00000100:00100000:2.0:1789762912.768825:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436056047963 time 436056154523 state 0x2 ref 0 00000100:00100000:0.0:1789762912.771254:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762912.771290:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015116032 00000100:00100000:2.0:1789762912.771302:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015116032 00000100:00100000:2.0:1789762912.771353:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436056047963 time 436056154523 state 0x0 ref 1 00000100:00100000:2.0:1789762912.771367:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436256154523 time 436056154523 state 0x1 ref 1 00000100:00100000:2.0:1789762912.952364:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436256154523 time 436256248013 state 0x0 ref 1 00000100:00100000:2.0:1789762912.952405:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015116032:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762912.952493:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 16777216, mtime: 0x6aad9d60, atime: 0x0, ctime: 0x6aad9d60, blocks: 0x8007 00010000:00010000:2.0:1789762912.952504:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0559e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.952538:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=436s v=(0 0) 00010000:00010000:2.0:1789762912.952545:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762912.952553:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0559e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.952569:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0559e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762912.966697:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968165 is committed 00002000:00010000:2.0:1789762912.966756:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 16777216 -> 17825792 00002000:00010000:2.0:1789762912.966761:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:2.0:1789762912.966774:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0559e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.966784:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0559e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.966791:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0559e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.966796:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762912.966799:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0559e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.966806:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0559e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.966822:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0559e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762912.966826:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b0559e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762912.966909:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015116032:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14520us (195653us total) trans 4294968165 rc 0/0 00000100:00100000:2.0:1789762912.966925:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1173 00000100:00100000:2.0:1789762912.966936:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436256154523 time 436256248013 state 0x2 ref 0 00000100:00100000:0.0:1789762912.969364:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762912.969440:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015116160 00000100:00100000:2.0:1789762912.969449:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015116160 00000100:00100000:2.0:1789762912.969489:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436256154523 time 436256248013 state 0x0 ref 1 00000100:00100000:2.0:1789762912.969501:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436456248013 time 436256248013 state 0x1 ref 1 00000100:00100000:2.0:1789762913.152454:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436456248013 time 436456343177 state 0x0 ref 1 00000100:00100000:2.0:1789762913.152503:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015116160:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:1.0:1789762913.152551:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762913.152569:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 17825792, mtime: 0x6aad9d60, atime: 0x0, ctime: 0x6aad9d60, blocks: 0x8807 00010000:00010000:2.0:1789762913.152578:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055a5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.152591:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=436s v=(0 0) 00010000:00010000:2.0:1789762913.152595:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762913.152600:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.152611:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762913.164405:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968166 is committed 00002000:00010000:2.0:1789762913.164455:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:2.0:1789762913.164460:0:14912:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762912 -> 1789762913 00002000:00010000:2.0:1789762913.164462:0:14912:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762912 -> 1789762913 00002000:00010000:2.0:1789762913.164464:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:2.0:1789762913.164474:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055a5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.164483:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055a5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.164490:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.164495:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762913.164498:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055a5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.164504:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055a5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.164535:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055a5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.164541:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055a5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762913.164630:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015116160:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12156us (195268us total) trans 4294968166 rc 0/0 00000100:00100000:2.0:1789762913.164639:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1174 00000100:00100000:2.0:1789762913.164650:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436456248013 time 436456343177 state 0x2 ref 0 00000100:00100000:3.0:1789762913.167117:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762913.167166:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015116288 00000100:00100000:1.0:1789762913.167175:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015116288 00000100:00100000:1.0:1789762913.167213:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436456248013 time 436456343177 state 0x0 ref 1 00000100:00100000:1.0:1789762913.167224:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436656343177 time 436456343177 state 0x1 ref 1 00010000:00010000:3.0:1789762913.207215:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762913.352563:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436656343177 time 436656456068 state 0x0 ref 1 00000100:00100000:2.0:1789762913.352594:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015116288:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762913.352647:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 18874368, mtime: 0x6aad9d61, atime: 0x0, ctime: 0x6aad9d61, blocks: 0x9007 00010000:00010000:2.0:1789762913.352657:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055ac lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352673:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=436s v=(0 0) 00010000:00010000:2.0:1789762913.352679:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762913.352685:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.352699:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762913.365481:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968167 is committed 00002000:00010000:2.0:1789762913.365501:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:2.0:1789762913.365506:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:2.0:1789762913.365531:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055ac lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.365548:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055ac lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.365554:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.365559:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762913.365562:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055ac lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.365569:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055ac lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.365583:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055ac lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.365588:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055ac lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762913.365656:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015116288:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13073us (198565us total) trans 4294968167 rc 0/0 00000100:00100000:2.0:1789762913.365666:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1175 00000100:00100000:2.0:1789762913.365675:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436656343177 time 436656456068 state 0x2 ref 0 00000100:00100000:3.0:1789762913.367972:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762913.368013:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015116416 00000100:00100000:2.0:1789762913.368027:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015116416 00000100:00100000:2.0:1789762913.368066:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436656343177 time 436656456068 state 0x0 ref 1 00000100:00100000:2.0:1789762913.368077:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436856456068 time 436656456068 state 0x1 ref 1 00000004:00080000:0.0:1789762913.408908:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 24 committed 00000001:00080000:0.0:1789762913.408931:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967596 is committed 00000004:00080000:0.0:1789762913.408937:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 29 committed 00000004:00080000:0.0:1789762913.408946:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 23 committed 00000001:00080000:0.0:1789762913.408968:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967604 is committed 00000001:00080000:0.0:1789762913.408991:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967605 is committed 00000100:00100000:0.0:1789762913.409025:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac26300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022189952:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789762913.409040:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc435c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022190080:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789762913.409049:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022190208:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789762913.409125:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762913.409158:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022190080 00000100:00100000:1.0:1789762913.409160:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762913.409166:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022190080 00000100:00100000:0.0:1789762913.409183:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022190208 00000100:00100000:0.0:1789762913.409189:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022190208 00000100:00100000:2.0:1789762913.409229:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:3.0:1789762913.409240:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:2.0:1789762913.409242:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022190080:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1789762913.409252:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022190208:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789762913.409260:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00002000:00080000:2.0:1789762913.409267:0:6881:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:23 count 1 00002000:00080000:3.0:1789762913.409272:0:6649:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:14 count 1 00000100:00100000:1.0:1789762913.409284:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022189952 00000100:00100000:1.0:1789762913.409290:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022189952 00002000:00010000:2.0:1789762913.409298:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x17:0x0] initial LVB size: 8388608, mtime: 0x6aad9d5b, atime: 0x0, ctime: 0x6aad9d5b, blocks: 0x4007 00002000:00010000:3.0:1789762913.409300:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 8388608, mtime: 0x6aad9d5b, atime: 0x0, ctime: 0x6aad9d5b, blocks: 0x4007 00010000:00010000:2.0:1789762913.409311:0:6881:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055b3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762913.409312:0:6649:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000055404050/0x416193e012b055ba lrc: 3/0,1 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762913.409323:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:1.0:1789762913.409340:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022189952:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1789762913.409344:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=436s v=(0 0) 00010000:00010000:3.0:1789762913.409344:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=436s v=(0 0) 00010000:00010000:2.0:1789762913.409355:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762913.409355:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00002000:00080000:1.0:1789762913.409357:0:6648:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:15 count 1 00010000:00010000:2.0:1789762913.409372:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762913.409372:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000055404050/0x416193e012b055ba lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762913.409382:0:6648:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 8388608, mtime: 0x6aad9d5c, atime: 0x0, ctime: 0x6aad9d5c, blocks: 0x4007 00010000:00010000:1.0:1789762913.409395:0:6648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a8f0f74b/0x416193e012b055c1 lrc: 3/0,1 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.409403:0:6881:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762913.409404:0:6649:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000055404050/0x416193e012b055ba lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.409406:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=436s v=(0 0) 00010000:00010000:1.0:1789762913.409417:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762913.409428:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a8f0f74b/0x416193e012b055c1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762913.409433:0:6649:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000055404050/0x416193e012b055ba lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.409434:0:6881:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055b3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.409439:0:6648:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a8f0f74b/0x416193e012b055c1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762913.409462:0:6649:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000055404050/0x416193e012b055ba lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.409464:0:6881:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762913.409465:0:6648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a8f0f74b/0x416193e012b055c1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762913.409493:0:6649:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762913.409497:0:6648:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a8f0f74b/0x416193e012b055c1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.409536:0:6881:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:1.0:1789762913.409546:0:6648:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:0.0:1789762913.409548:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1789762913.409556:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000055404050/0x416193e012b055ba lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409566:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762913.409571:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000055404050/0x416193e012b055ba lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409581:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000055404050/0x416193e012b055ba lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409600:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000055404050/0x416193e012b055ba lrc: 1/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409619:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000055404050/0x416193e012b055ba lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409638:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1789762913.409643:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409650:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762913.409654:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055b3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409662:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055b3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409673:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055b3 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409680:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000032976790/0x416193e012b055b3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409690:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789762913.409694:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a8f0f74b/0x416193e012b055c1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409699:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4ca00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762913.409710:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a8f0f74b/0x416193e012b055c1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409718:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a8f0f74b/0x416193e012b055c1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409728:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a8f0f74b/0x416193e012b055c1 lrc: 1/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409736:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a8f0f74b/0x416193e012b055c1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762913.409756:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762913.409759:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762913.409762:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762913.409770:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762913.410144:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762913.410469:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022190080:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1230us (1343us total) trans 4294968168 rc 0/0 00000100:00100000:2.0:1789762913.410477:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:3.0:1789762913.410477:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3271:x1876702022190208:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1227us (1317us total) trans 4294967433 rc 0/0 00000100:00100000:3.0:1789762913.410489:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:1.0:1789762913.410494:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+8:3273:x1876702022189952:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1154us (1231us total) trans 4294967434 rc 0/0 00000004:00080000:0.0:1789762913.410505:0:3274:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebc435c00/2, rc 0, transno 4294968168 00000100:00100000:1.0:1789762913.410505:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00080000:2.0:1789762913.410506:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789762913.410513:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762913.410539:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc435c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022190080:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1789762913.410560:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebac25180/2, rc 0, transno 4294967433 00000004:00080000:0.0:1789762913.410563:0:3273:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebac26300/2, rc 0, transno 4294967434 00000100:00100000:1.0:1789762913.410566:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022190208:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789762913.410567:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac26300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022189952:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789762913.552629:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436856456068 time 436856523502 state 0x0 ref 1 00000100:00100000:2.0:1789762913.552655:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015116416:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762913.552706:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 19922944, mtime: 0x6aad9d61, atime: 0x0, ctime: 0x6aad9d61, blocks: 0x9807 00010000:00010000:2.0:1789762913.552713:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.552727:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=436s v=(0 0) 00010000:00010000:2.0:1789762913.552731:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762913.552736:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.552747:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762913.565227:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968169 is committed 00002000:00010000:2.0:1789762913.565235:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 19922944 -> 20971520 00000001:00080000:0.0:1789762913.565238:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968168 is committed 00002000:00010000:2.0:1789762913.565241:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:2.0:1789762913.565254:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.565272:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.565279:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.565284:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762913.565287:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.565293:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.565309:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.565316:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762913.565389:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015116416:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12742us (197416us total) trans 4294968169 rc 0/0 00000100:00100000:2.0:1789762913.565399:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1176 00000100:00100000:2.0:1789762913.565407:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436856456068 time 436856523502 state 0x2 ref 0 00000100:00100000:3.0:1789762913.567441:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762913.567474:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015116544 00000100:00100000:2.0:1789762913.567488:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015116544 00000100:00100000:2.0:1789762913.567538:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 436856456068 time 436856523502 state 0x0 ref 1 00000100:00100000:2.0:1789762913.567549:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437056523502 time 436856523502 state 0x1 ref 1 00000100:00100000:2.0:1789762913.752715:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437056523502 time 437056607731 state 0x0 ref 1 00000100:00100000:2.0:1789762913.752744:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe72a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015116544:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762913.752800:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 20971520, mtime: 0x6aad9d61, atime: 0x0, ctime: 0x6aad9d61, blocks: 0xa007 00010000:00010000:2.0:1789762913.752808:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.752824:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=437s v=(0 0) 00010000:00010000:2.0:1789762913.752828:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762913.752834:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.752849:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762913.766942:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968170 is committed 00002000:00010000:2.0:1789762913.766970:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:2.0:1789762913.766976:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:2.0:1789762913.766988:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.767001:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.767022:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.767031:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762913.767036:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.767047:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055cf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.767068:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055cf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.767075:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762913.767185:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe72a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015116544:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14451us (199742us total) trans 4294968170 rc 0/0 00000100:00100000:2.0:1789762913.767194:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1177 00000100:00100000:2.0:1789762913.767204:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437056523502 time 437056607731 state 0x2 ref 0 00000100:00100000:0.0:1789762913.769411:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762913.769459:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015116672 00000100:00100000:1.0:1789762913.769467:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015116672 00000100:00100000:1.0:1789762913.769508:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437056523502 time 437056607731 state 0x0 ref 1 00000100:00100000:1.0:1789762913.769536:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437256607731 time 437056607731 state 0x1 ref 1 00000100:00100000:2.0:1789762913.952863:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437256607731 time 437256751136 state 0x0 ref 1 00000100:00100000:2.0:1789762913.952897:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015116672:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762913.952975:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 22020096, mtime: 0x6aad9d61, atime: 0x0, ctime: 0x6aad9d61, blocks: 0xa807 00010000:00010000:2.0:1789762913.952985:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055d6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.953004:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=437s v=(0 0) 00010000:00010000:2.0:1789762913.953009:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762913.953016:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.953030:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762913.972755:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968171 is committed 00002000:00010000:2.0:1789762913.972806:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 22020096 -> 23068672 00002000:00010000:2.0:1789762913.972820:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:2.0:1789762913.972844:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055d6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.972870:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055d6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.972883:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.972893:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762913.972899:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055d6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.972935:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055d6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.972959:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055d6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762913.972970:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055d6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762913.973131:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015116672:12345-192.168.203.48@tcp:4:dd.500 Request processed in 20202us (203674us total) trans 4294968171 rc 0/0 00000100:00100000:2.0:1789762913.973145:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1178 00000100:00100000:2.0:1789762913.973159:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437256607731 time 437256751136 state 0x2 ref 0 00000100:00100000:3.0:1789762913.977195:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762913.977264:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015116800 00000100:00100000:2.0:1789762913.977279:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015116800 00000100:00100000:2.0:1789762913.977367:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437256607731 time 437256751136 state 0x0 ref 1 00000100:00100000:2.0:1789762913.977395:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437456751136 time 437256751136 state 0x1 ref 1 00000100:00100000:2.0:1789762914.152904:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437456751136 time 437456799623 state 0x0 ref 1 00000100:00100000:2.0:1789762914.152933:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015116800:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762914.152990:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 23068672, mtime: 0x6aad9d61, atime: 0x0, ctime: 0x6aad9d61, blocks: 0xb007 00010000:00010000:2.0:1789762914.153009:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055dd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.153022:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=437s v=(0 0) 00010000:00010000:2.0:1789762914.153026:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762914.153031:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.153042:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762914.164995:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968172 is committed 00002000:00010000:2.0:1789762914.165057:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:2.0:1789762914.165062:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762913 -> 1789762914 00002000:00010000:2.0:1789762914.165064:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762913 -> 1789762914 00002000:00010000:2.0:1789762914.165066:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:2.0:1789762914.165075:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055dd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.165108:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055dd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.165119:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.165128:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762914.165133:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055dd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.165155:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055dd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.165171:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055dd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.165177:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055dd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762914.165248:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015116800:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12326us (188058us total) trans 4294968172 rc 0/0 00000100:00100000:2.0:1789762914.165258:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1179 00000100:00100000:2.0:1789762914.165267:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437456751136 time 437456799623 state 0x2 ref 0 00000100:00100000:0.0:1789762914.167460:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762914.167505:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015116928 00000100:00100000:2.0:1789762914.167525:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015116928 00000100:00100000:2.0:1789762914.167572:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437456751136 time 437456799623 state 0x0 ref 1 00000100:00100000:2.0:1789762914.167584:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437656799623 time 437456799623 state 0x1 ref 1 00010000:00010000:3.0:1789762914.231253:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762914.352958:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437656799623 time 437656854168 state 0x0 ref 1 00000100:00100000:2.0:1789762914.352986:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015116928:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762914.353037:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 24117248, mtime: 0x6aad9d62, atime: 0x0, ctime: 0x6aad9d62, blocks: 0xb807 00010000:00010000:2.0:1789762914.353056:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055e4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.353069:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=437s v=(0 0) 00010000:00010000:2.0:1789762914.353073:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762914.353077:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.353117:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762914.364328:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968173 is committed 00002000:00010000:2.0:1789762914.364352:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 24117248 -> 25165824 00002000:00010000:2.0:1789762914.364357:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:2.0:1789762914.364367:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055e4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.364376:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055e4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.364383:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.364388:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762914.364392:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055e4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.364399:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055e4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.364421:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055e4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.364427:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055e4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762914.364488:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015116928:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11512us (197028us total) trans 4294968173 rc 0/0 00000100:00100000:2.0:1789762914.364496:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1180 00000100:00100000:2.0:1789762914.364505:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437656799623 time 437656854168 state 0x2 ref 0 00000100:00100000:0.0:1789762914.366778:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762914.366832:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015117056 00000100:00100000:1.0:1789762914.366840:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015117056 00000100:00100000:1.0:1789762914.366880:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437656799623 time 437656854168 state 0x0 ref 1 00000100:00100000:1.0:1789762914.366891:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437856854168 time 437656854168 state 0x1 ref 1 00000100:00100000:2.0:1789762914.553052:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437856854168 time 437856947365 state 0x0 ref 1 00000100:00100000:2.0:1789762914.553079:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015117056:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762914.553158:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 25165824, mtime: 0x6aad9d62, atime: 0x0, ctime: 0x6aad9d62, blocks: 0xc007 00010000:00010000:2.0:1789762914.553165:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055eb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.553179:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=437s v=(0 0) 00010000:00010000:2.0:1789762914.553183:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762914.553187:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.553198:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762914.565530:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968174 is committed 00002000:00010000:2.0:1789762914.565541:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:2.0:1789762914.565546:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:2.0:1789762914.565556:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055eb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.565567:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055eb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.565573:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.565578:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762914.565582:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055eb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.565589:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055eb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.565604:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055eb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.565609:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055eb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762914.565674:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015117056:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12602us (198894us total) trans 4294968174 rc 0/0 00000100:00100000:2.0:1789762914.565690:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1181 00000100:00100000:2.0:1789762914.565699:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437856854168 time 437856947365 state 0x2 ref 0 00000100:00100000:2.0:1789762914.567886:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762914.567928:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015117184 00000100:00100000:3.0:1789762914.567936:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015117184 00000100:00100000:3.0:1789762914.567977:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 437856854168 time 437856947365 state 0x0 ref 1 00000100:00100000:3.0:1789762914.567988:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438056947365 time 437856947365 state 0x1 ref 1 00000100:00100000:2.0:1789762914.753152:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438056947365 time 438057042528 state 0x0 ref 1 00000100:00100000:2.0:1789762914.753183:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015117184:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762914.753236:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 26214400, mtime: 0x6aad9d62, atime: 0x0, ctime: 0x6aad9d62, blocks: 0xc807 00010000:00010000:2.0:1789762914.753244:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055f2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.753258:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=438s v=(0 0) 00010000:00010000:2.0:1789762914.753261:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762914.753266:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.753278:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762914.764903:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968175 is committed 00002000:00010000:1.0:1789762914.764949:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 26214400 -> 27262976 00002000:00010000:1.0:1789762914.764954:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:1.0:1789762914.764963:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055f2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.764972:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055f2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.764979:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.764984:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762914.764987:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055f2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.764993:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055f2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.765006:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055f2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.765011:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b055f2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762914.765074:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015117184:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11903us (197187us total) trans 4294968175 rc 0/0 00000100:00100000:1.0:1789762914.765081:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1182 00000100:00100000:1.0:1789762914.765121:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438056947365 time 438057042528 state 0x2 ref 0 00000100:00100000:1.0:1789762914.767260:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762914.767312:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015117312 00000100:00100000:2.0:1789762914.767322:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015117312 00000100:00100000:2.0:1789762914.767363:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438056947365 time 438057042528 state 0x0 ref 1 00000100:00100000:2.0:1789762914.767374:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438257042528 time 438057042528 state 0x1 ref 1 00000100:00100000:2.0:1789762914.953224:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438257042528 time 438257117698 state 0x0 ref 1 00000100:00100000:2.0:1789762914.953252:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015117312:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762914.953316:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 27262976, mtime: 0x6aad9d62, atime: 0x0, ctime: 0x6aad9d62, blocks: 0xd007 00010000:00010000:2.0:1789762914.953323:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b055f9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.953337:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=438s v=(0 0) 00010000:00010000:2.0:1789762914.953341:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762914.953345:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b055f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762914.953356:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b055f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762914.966375:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968176 is committed 00002000:00010000:1.0:1789762914.966401:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:1.0:1789762914.966406:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:1.0:1789762914.966416:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b055f9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.966426:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b055f9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.966432:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b055f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.966438:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762914.966441:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b055f9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.966448:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b055f9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.966461:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b055f9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762914.966466:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ce52c272/0x416193e012b055f9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762914.966545:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015117312:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13302us (199281us total) trans 4294968176 rc 0/0 00000100:00100000:1.0:1789762914.966569:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1183 00000100:00100000:1.0:1789762914.966580:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438257042528 time 438257117698 state 0x2 ref 0 00000100:00100000:0.0:1789762914.969154:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762914.969193:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015117440 00000100:00100000:1.0:1789762914.969201:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015117440 00000100:00100000:1.0:1789762914.969239:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438257042528 time 438257117698 state 0x0 ref 1 00000100:00100000:1.0:1789762914.969251:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438457117698 time 438257117698 state 0x1 ref 1 00000100:00100000:2.0:1789762915.153342:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438457117698 time 438457232703 state 0x0 ref 1 00000100:00100000:2.0:1789762915.153390:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015117440:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:1.0:1789762915.153409:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789762915.153451:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 28311552, mtime: 0x6aad9d62, atime: 0x0, ctime: 0x6aad9d62, blocks: 0xd807 00010000:00010000:2.0:1789762915.153470:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b05600 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762915.153488:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=438s v=(0 0) 00010000:00010000:2.0:1789762915.153494:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762915.153530:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b05600 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762915.153547:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b05600 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762915.164831:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968177 is committed 00002000:00010000:0.0:1789762915.164865:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:0.0:1789762915.164868:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762914 -> 1789762915 00002000:00010000:0.0:1789762915.164870:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762914 -> 1789762915 00002000:00010000:0.0:1789762915.164872:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 55303 -> 57351 00010000:00010000:0.0:1789762915.164881:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b05600 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.164891:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b05600 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.164897:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b05600 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.164902:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762915.164906:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b05600 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.164912:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b05600 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.164927:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b05600 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.164943:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003a595394/0x416193e012b05600 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762915.165008:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015117440:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11644us (195852us total) trans 4294968177 rc 0/0 00000100:00100000:0.0:1789762915.165017:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1184 00000100:00100000:0.0:1789762915.165026:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438457117698 time 438457232703 state 0x2 ref 0 00000100:00100000:0.0:1789762915.167352:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762915.167386:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015117568 00000100:00100000:3.0:1789762915.167395:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015117568 00000100:00100000:3.0:1789762915.167436:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438457117698 time 438457232703 state 0x0 ref 1 00000100:00100000:3.0:1789762915.167448:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438657232703 time 438457232703 state 0x1 ref 1 00010000:00010000:0.0:1789762915.255223:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762915.353417:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438657232703 time 438657311268 state 0x0 ref 1 00000100:00100000:1.0:1789762915.353462:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb27480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015117568:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762915.353565:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 29360128, mtime: 0x6aad9d63, atime: 0x0, ctime: 0x6aad9d63, blocks: 0xe007 00010000:00010000:1.0:1789762915.353573:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05607 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762915.353587:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=438s v=(0 0) 00010000:00010000:1.0:1789762915.353590:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762915.353595:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05607 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762915.353607:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05607 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762915.367455:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968178 is committed 00002000:00010000:0.0:1789762915.367535:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:0.0:1789762915.367539:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 57351 -> 59399 00010000:00010000:0.0:1789762915.367551:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05607 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.367560:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05607 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.367567:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05607 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.367572:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4ce00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762915.367575:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05607 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.367593:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05607 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.367608:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05607 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.367614:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000761492db/0x416193e012b05607 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762915.367682:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb27480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015117568:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14231us (200328us total) trans 4294968178 rc 0/0 00000100:00100000:0.0:1789762915.367691:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1185 00000100:00100000:0.0:1789762915.367701:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438657232703 time 438657311268 state 0x2 ref 0 00000100:00100000:3.0:1789762915.370259:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762915.370356:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015117696 00000100:00100000:2.0:1789762915.370364:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015117696 00000100:00100000:2.0:1789762915.370405:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438657232703 time 438657311268 state 0x0 ref 1 00000100:00100000:2.0:1789762915.370416:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438857311268 time 438657311268 state 0x1 ref 1 00000100:00100000:1.0:1789762915.426401:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762915.426433:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762915.426467:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015117824 00000100:00100000:2.0:1789762915.426478:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015117824 00000100:00100000:1.0:1789762915.426484:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762915.426523:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015117952 00000100:00100000:3.0:1789762915.426530:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015117952 00000100:00100000:1.0:1789762915.426550:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762915.426552:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015118208 00000100:00100000:0.0:1789762915.426559:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015118208 00000100:00100000:2.0:1789762915.426565:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 344 00000100:00100000:2.0:1789762915.426580:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015117824:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:1.0:1789762915.426586:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762915.426591:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 347 00000100:00100000:3.0:1789762915.426604:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702015117952:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762915.426610:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 348 00000100:00100000:0.0:1789762915.426620:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015118208:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762915.426651:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015117824:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 74us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762915.426652:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015118080 00000100:00100000:3.0:1789762915.426654:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2366:x1876702015117952:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 53us (218us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762915.426659:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015118080 00000100:00100000:0.0:1789762915.426663:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015118208:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 41us (179us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762915.426672:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 344 00000100:00100000:3.0:1789762915.426673:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 347 00000100:00100000:0.0:1789762915.426683:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 348 00000100:00100000:1.0:1789762915.426712:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 173 00000100:00100000:1.0:1789762915.426725:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015118080:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:0.0:1789762915.426741:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762915.426780:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015118080:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 58us (230us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762915.426788:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 173 00000004:00080000:0.0:1789762915.447270:0:6868:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 438751180879ns, next at 439751178326ns, rc = 0 00000100:00100000:2.0:1789762915.447273:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb24700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022190336:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762915.447328:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762915.447383:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022190336 00000100:00100000:3.0:1789762915.447392:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022190336 00000100:00080000:0.0:1789762915.447438:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789762915.447456:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 145 00000100:00100000:3.0:1789762915.447469:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022190336:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789762915.447492:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 525926400 pending 0 free 3830448128 avail 3818913792 00000100:00100000:3.0:1789762915.447557:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022190336:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (228us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762915.447563:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00000100:00100000:2.0:1789762915.447637:0:3272:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294968168 gen 2 00000004:00080000:2.0:1789762915.447642:0:3272:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebc435c00, transno 4294968168 00000100:00100000:2.0:1789762915.447659:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb24700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022190336:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762915.553549:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438857311268 time 438857444178 state 0x0 ref 1 00000100:00100000:1.0:1789762915.553576:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015117696:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762915.553629:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 30408704, mtime: 0x6aad9d63, atime: 0x0, ctime: 0x6aad9d63, blocks: 0xe807 00010000:00010000:1.0:1789762915.553636:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b0560e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762915.553649:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=438s v=(0 0) 00010000:00010000:1.0:1789762915.553664:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762915.553669:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b0560e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762915.553683:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b0560e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762915.566201:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968179 is committed 00002000:00010000:0.0:1789762915.566214:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 30408704 -> 31457280 00002000:00010000:0.0:1789762915.566220:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 59399 -> 61447 00010000:00010000:0.0:1789762915.566232:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b0560e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.566242:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b0560e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.566249:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b0560e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.566254:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762915.566268:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b0560e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.566275:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b0560e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.566289:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b0560e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.566294:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b0560e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762915.566371:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015117696:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12804us (196111us total) trans 4294968179 rc 0/0 00000100:00100000:0.0:1789762915.566380:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1186 00000100:00100000:0.0:1789762915.566397:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438857311268 time 438857444178 state 0x2 ref 0 00000100:00100000:2.0:1789762915.568990:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762915.569042:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015118336 00000100:00100000:1.0:1789762915.569050:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015118336 00000100:00100000:1.0:1789762915.569117:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 438857311268 time 438857444178 state 0x0 ref 1 00000100:00100000:1.0:1789762915.569128:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439057444178 time 438857444178 state 0x1 ref 1 00000100:00100000:0.0:1789762915.753654:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439057444178 time 439057544347 state 0x0 ref 1 00000100:00100000:0.0:1789762915.753684:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015118336:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762915.753734:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 31457280, mtime: 0x6aad9d63, atime: 0x0, ctime: 0x6aad9d63, blocks: 0xf007 00010000:00010000:0.0:1789762915.753741:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05615 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.753755:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=439s v=(0 0) 00010000:00010000:0.0:1789762915.753758:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762915.753763:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05615 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.753775:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05615 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762915.765263:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 31457280 -> 32505856 00000001:00080000:1.0:1789762915.765264:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968180 is committed 00002000:00010000:2.0:1789762915.765274:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 61447 -> 63495 00010000:00010000:2.0:1789762915.765285:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05615 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762915.765295:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05615 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762915.765302:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05615 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762915.765307:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762915.765310:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05615 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762915.765316:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05615 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762915.765331:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05615 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762915.765337:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05615 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762915.765401:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015118336:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11727us (196410us total) trans 4294968180 rc 0/0 00000100:00100000:2.0:1789762915.765409:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1187 00000100:00100000:2.0:1789762915.765418:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439057444178 time 439057544347 state 0x2 ref 0 00000100:00100000:3.0:1789762915.767720:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762915.767757:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015118464 00000100:00100000:2.0:1789762915.767765:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015118464 00000100:00100000:2.0:1789762915.767804:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439057444178 time 439057544347 state 0x0 ref 1 00000100:00100000:2.0:1789762915.767816:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439257544347 time 439057544347 state 0x1 ref 1 00000100:00100000:0.0:1789762915.953746:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439257544347 time 439257641547 state 0x0 ref 1 00000100:00100000:0.0:1789762915.953771:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015118464:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762915.953820:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 32505856, mtime: 0x6aad9d63, atime: 0x0, ctime: 0x6aad9d63, blocks: 0xf807 00010000:00010000:0.0:1789762915.953828:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0561c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.953841:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=439s v=(0 0) 00010000:00010000:0.0:1789762915.953858:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762915.953863:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0561c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.953874:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0561c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762915.960198:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac26d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022190464:0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.960202:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb25c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022190592:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762915.960210:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac26680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022190720:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.960250:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac27100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022190848:0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.960261:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762915.960282:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762915.960332:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762915.960337:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022190592 00000100:00100000:3.0:1789762915.960337:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac24000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022191104:0@lo:400:kworker.0 00000100:00100000:1.0:1789762915.960338:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac27800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022191232:0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.960346:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022190592 00000100:00100000:3.0:1789762915.960383:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762915.960388:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762915.960407:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:3.0:1789762915.960412:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789762915.960418:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3274:x1876702022190592:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762915.960422:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac24a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022190976:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.960434:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022190720 00000100:00100000:1.0:1789762915.960436:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022190848 00000100:00100000:2.0:1789762915.960439:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022190720 00000100:00100000:1.0:1789762915.960442:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022190848 00002000:00000024:0.0:1789762915.960443:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3843031040 avail 3840933888 00000100:00100000:3.0:1789762915.960453:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762915.960480:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022191104 00000100:00100000:2.0:1789762915.960485:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022191104 00000100:00100000:1.0:1789762915.960510:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 349 00000100:00100000:0.0:1789762915.960512:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3274:x1876702022190592:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762915.960522:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022190720:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.960524:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00100000:3.0:1789762915.960528:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb25c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022190592:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762915.960534:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 350 00000100:00100000:2.0:1789762915.960549:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022190848:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762915.960550:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789762915.960556:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762915.960566:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022191232 00000100:00100000:3.0:1789762915.960571:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022191232 00000100:00100000:1.0:1789762915.960573:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022190720:12345-0@lo:400:kworker.0 Request processed in 53us (290us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762915.960577:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 351 00000100:00100000:1.0:1789762915.960587:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 349 00000100:00100000:0.0:1789762915.960593:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022191104:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.960610:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022190976 00000100:00100000:2.0:1789762915.960611:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022190848:12345-0@lo:400:kworker.0 Request processed in 65us (280us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762915.960614:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022190976 00000100:00100000:2.0:1789762915.960623:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 350 00000100:00100000:1.0:1789762915.960628:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:1.0:1789762915.960640:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+7:3271:x1876702022191232:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.960646:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022191104:12345-0@lo:400:kworker.0 Request processed in 57us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762915.960655:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 351 00000100:00100000:3.0:1789762915.960657:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:2.0:1789762915.960669:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac27100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022190848:0@lo:400:kworker.0 00000100:00100000:3.0:1789762915.960671:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022190976:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762915.960682:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+7:3271:x1876702022191232:12345-0@lo:400:kworker.0 Request processed in 44us (295us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762915.960689:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00100000:0.0:1789762915.960712:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762915.960721:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022190976:12345-0@lo:400:kworker.0 Request processed in 55us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762915.960727:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:1.0:1789762915.960735:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac27800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022191232:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.960749:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022190464 00000100:00100000:2.0:1789762915.960754:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022190464 00000100:00100000:1.0:1789762915.960758:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac26680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022190720:0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.960795:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac24a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022190976:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.960795:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 345 00000100:00100000:2.0:1789762915.960810:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850db450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022190464:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762915.960817:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac24000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022191104:0@lo:400:kworker.0 00000100:00100000:2.0:1789762915.960857:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850db450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022190464:12345-0@lo:400:kworker.0 Request processed in 50us (145us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762915.960865:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 345 00000100:00100000:3.0:1789762915.960874:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac26d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022190464:0@lo:400:kworker.0 00000001:00080000:1.0:1789762915.967946:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968181 is committed 00002000:00010000:0.0:1789762915.967964:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:0.0:1789762915.967969:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 63495 -> 65543 00010000:00010000:0.0:1789762915.967979:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0561c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.967988:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0561c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.967995:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0561c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.968000:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762915.968004:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0561c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.968021:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0561c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.968035:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0561c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762915.968040:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0561c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762915.968113:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015118464:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14326us (200367us total) trans 4294968181 rc 0/0 00000100:00100000:0.0:1789762915.968122:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1188 00000100:00100000:0.0:1789762915.968141:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439257544347 time 439257641547 state 0x2 ref 0 00000100:00100000:3.0:1789762915.970577:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762915.970637:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015118592 00000100:00100000:2.0:1789762915.970643:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015118592 00000100:00100000:2.0:1789762915.970673:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439257544347 time 439257641547 state 0x0 ref 1 00000100:00100000:2.0:1789762915.970684:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439457641547 time 439257641547 state 0x1 ref 1 00000100:00100000:0.0:1789762916.153855:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439457641547 time 439457738478 state 0x0 ref 1 00000100:00100000:0.0:1789762916.153881:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015118592:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762916.153933:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 33554432, mtime: 0x6aad9d63, atime: 0x0, ctime: 0x6aad9d63, blocks: 0x10007 00010000:00010000:0.0:1789762916.153940:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05623 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.153953:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=439s v=(0 0) 00010000:00010000:0.0:1789762916.153957:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762916.153962:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05623 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.153973:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05623 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762916.166695:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968182 is committed 00002000:00010000:0.0:1789762916.166708:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:0.0:1789762916.166713:0:14912:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762915 -> 1789762916 00002000:00010000:0.0:1789762916.166727:0:14912:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762915 -> 1789762916 00002000:00010000:0.0:1789762916.166729:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 65543 -> 67591 00010000:00010000:0.0:1789762916.166738:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05623 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.166747:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05623 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.166764:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05623 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.166770:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762916.166773:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05623 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.166780:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05623 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.166812:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05623 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.166818:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05623 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762916.166902:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015118592:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13018us (196312us total) trans 4294968182 rc 0/0 00000100:00100000:0.0:1789762916.166912:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1189 00000100:00100000:0.0:1789762916.166924:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439457641547 time 439457738478 state 0x2 ref 0 00000100:00100000:3.0:1789762916.169537:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762916.169582:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015118720 00000100:00100000:1.0:1789762916.169602:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015118720 00000100:00100000:1.0:1789762916.169641:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439457641547 time 439457738478 state 0x0 ref 1 00000100:00100000:1.0:1789762916.169652:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439657738478 time 439457738478 state 0x1 ref 1 00010000:00010000:0.0:1789762916.279186:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762916.353934:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439657738478 time 439657823947 state 0x0 ref 1 00000100:00100000:2.0:1789762916.353962:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015118720:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762916.354020:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 34603008, mtime: 0x6aad9d64, atime: 0x0, ctime: 0x6aad9d64, blocks: 0x10807 00010000:00010000:2.0:1789762916.354031:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b0562a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762916.354049:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=439s v=(0 0) 00010000:00010000:2.0:1789762916.354054:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762916.354061:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b0562a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762916.354075:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b0562a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762916.365790:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968183 is committed 00002000:00010000:0.0:1789762916.365808:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:0.0:1789762916.365814:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 67591 -> 69639 00010000:00010000:0.0:1789762916.365825:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b0562a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.365841:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b0562a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.365848:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b0562a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.365853:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762916.365856:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b0562a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.365863:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b0562a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.365878:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b0562a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.365882:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e673f84f/0x416193e012b0562a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762916.365943:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015118720:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11988us (196404us total) trans 4294968183 rc 0/0 00000100:00100000:0.0:1789762916.365951:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1190 00000100:00100000:0.0:1789762916.365961:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439657738478 time 439657823947 state 0x2 ref 0 00000100:00100000:3.0:1789762916.368136:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762916.368174:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015118848 00000100:00100000:2.0:1789762916.368182:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015118848 00000100:00100000:2.0:1789762916.368220:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439657738478 time 439657823947 state 0x0 ref 1 00000100:00100000:2.0:1789762916.368232:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439857823947 time 439657823947 state 0x1 ref 1 00000100:00100000:0.0:1789762916.554048:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439857823947 time 439857937915 state 0x0 ref 1 00000100:00100000:0.0:1789762916.554154:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015118848:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762916.554211:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 35651584, mtime: 0x6aad9d64, atime: 0x0, ctime: 0x6aad9d64, blocks: 0x11007 00010000:00010000:0.0:1789762916.554219:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05631 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.554233:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=439s v=(0 0) 00010000:00010000:0.0:1789762916.554237:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762916.554241:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05631 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.554255:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05631 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762916.567291:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968184 is committed 00002000:00010000:0.0:1789762916.567340:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 35651584 -> 36700160 00002000:00010000:0.0:1789762916.567345:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 69639 -> 71687 00010000:00010000:0.0:1789762916.567354:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05631 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.567364:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05631 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.567371:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05631 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.567375:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762916.567385:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05631 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.567392:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05631 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.567406:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05631 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.567411:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05631 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762916.567473:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015118848:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13387us (199336us total) trans 4294968184 rc 0/0 00000100:00100000:0.0:1789762916.567482:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1191 00000100:00100000:0.0:1789762916.567491:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439857823947 time 439857937915 state 0x2 ref 0 00000100:00100000:3.0:1789762916.569896:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762916.569949:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015118976 00000100:00100000:2.0:1789762916.569958:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015118976 00000100:00100000:2.0:1789762916.570010:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 439857823947 time 439857937915 state 0x0 ref 1 00000100:00100000:2.0:1789762916.570025:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440057937915 time 439857937915 state 0x1 ref 1 00000100:00100000:0.0:1789762916.754137:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440057937915 time 440058032367 state 0x0 ref 1 00000100:00100000:0.0:1789762916.754180:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015118976:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762916.754241:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 36700160, mtime: 0x6aad9d64, atime: 0x0, ctime: 0x6aad9d64, blocks: 0x11807 00010000:00010000:0.0:1789762916.754249:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05638 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.754263:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=440s v=(0 0) 00010000:00010000:0.0:1789762916.754278:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762916.754284:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05638 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.754297:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05638 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762916.767177:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 36700160 -> 37748736 00000001:00080000:1.0:1789762916.767182:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968185 is committed 00002000:00010000:0.0:1789762916.767183:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 71687 -> 73735 00010000:00010000:0.0:1789762916.767196:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05638 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.767206:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05638 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.767212:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05638 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.767217:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762916.767221:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05638 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.767228:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05638 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.767242:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05638 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.767258:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b05638 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762916.767319:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015118976:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13164us (197423us total) trans 4294968185 rc 0/0 00000100:00100000:0.0:1789762916.767327:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1192 00000100:00100000:0.0:1789762916.767335:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440057937915 time 440058032367 state 0x2 ref 0 00000100:00100000:1.0:1789762916.769953:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762916.770001:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015119104 00000100:00100000:0.0:1789762916.770010:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015119104 00000100:00100000:0.0:1789762916.770054:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440057937915 time 440058032367 state 0x0 ref 1 00000100:00100000:0.0:1789762916.770065:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440258032367 time 440058032367 state 0x1 ref 1 00000100:00100000:0.0:1789762916.954242:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440258032367 time 440258132725 state 0x0 ref 1 00000100:00100000:0.0:1789762916.954283:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015119104:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762916.954346:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 37748736, mtime: 0x6aad9d64, atime: 0x0, ctime: 0x6aad9d64, blocks: 0x12007 00010000:00010000:0.0:1789762916.954354:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0563f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.954367:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=440s v=(0 0) 00010000:00010000:0.0:1789762916.954371:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762916.954388:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0563f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762916.954400:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0563f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762916.968278:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968186 is committed 00002000:00010000:2.0:1789762916.968295:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:2.0:1789762916.968300:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 73735 -> 75783 00010000:00010000:2.0:1789762916.968312:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0563f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762916.968334:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0563f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762916.968341:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0563f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762916.968346:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762916.968350:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0563f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762916.968357:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0563f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762916.968371:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0563f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762916.968387:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b62ed32d/0x416193e012b0563f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762916.968463:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015119104:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14201us (198508us total) trans 4294968186 rc 0/0 00000100:00100000:2.0:1789762916.968471:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1193 00000100:00100000:2.0:1789762916.968506:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440258032367 time 440258132725 state 0x2 ref 0 00000100:00100000:3.0:1789762916.970991:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762916.971028:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015119232 00000100:00100000:2.0:1789762916.971037:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015119232 00000100:00100000:2.0:1789762916.971075:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440258032367 time 440258132725 state 0x0 ref 1 00000100:00100000:2.0:1789762916.971102:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440458132725 time 440258132725 state 0x1 ref 1 00000100:00100000:0.0:1789762917.154414:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440458132725 time 440458273502 state 0x0 ref 1 00000100:00100000:0.0:1789762917.154487:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015119232:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:2.0:1789762917.154551:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789762917.154585:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 38797312, mtime: 0x6aad9d64, atime: 0x0, ctime: 0x6aad9d64, blocks: 0x12807 00010000:00010000:0.0:1789762917.154595:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05646 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762917.154612:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=440s v=(0 0) 00010000:00010000:0.0:1789762917.154617:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762917.154623:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05646 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762917.154636:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05646 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762917.169146:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968187 is committed 00002000:00010000:2.0:1789762917.169181:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:2.0:1789762917.169187:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762916 -> 1789762917 00002000:00010000:2.0:1789762917.169190:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762916 -> 1789762917 00002000:00010000:2.0:1789762917.169192:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 75783 -> 77831 00010000:00010000:2.0:1789762917.169205:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05646 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.169217:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05646 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.169241:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05646 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.169248:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762917.169252:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05646 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.169260:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05646 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.169278:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05646 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.169285:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05646 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762917.169402:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015119232:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14952us (198404us total) trans 4294968187 rc 0/0 00000100:00100000:2.0:1789762917.169417:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1194 00000100:00100000:2.0:1789762917.169449:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440458132725 time 440458273502 state 0x2 ref 0 00000100:00100000:3.0:1789762917.172992:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762917.173073:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015119360 00000100:00100000:2.0:1789762917.173108:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015119360 00000100:00100000:2.0:1789762917.173167:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440458132725 time 440458273502 state 0x0 ref 1 00000100:00100000:2.0:1789762917.173201:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440658273502 time 440458273502 state 0x1 ref 1 00010000:00010000:0.0:1789762917.303208:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762917.354472:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440658273502 time 440658368144 state 0x0 ref 1 00000100:00100000:0.0:1789762917.354510:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015119360:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762917.354572:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 39845888, mtime: 0x6aad9d65, atime: 0x0, ctime: 0x6aad9d65, blocks: 0x13007 00010000:00010000:0.0:1789762917.354580:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0564d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762917.354593:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=440s v=(0 0) 00010000:00010000:0.0:1789762917.354597:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762917.354602:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0564d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762917.354613:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0564d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762917.366815:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 39845888 -> 40894464 00000001:00080000:2.0:1789762917.366820:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968188 is committed 00002000:00010000:1.0:1789762917.366821:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 77831 -> 79879 00010000:00010000:1.0:1789762917.366835:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0564d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.366846:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0564d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.366858:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0564d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.366863:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762917.366867:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0564d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.366873:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0564d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.366889:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0564d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.366894:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0564d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762917.366966:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015119360:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12474us (193975us total) trans 4294968188 rc 0/0 00000100:00100000:1.0:1789762917.366978:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1195 00000100:00100000:1.0:1789762917.366992:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440658273502 time 440658368144 state 0x2 ref 0 00000100:00100000:1.0:1789762917.369266:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762917.369316:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015119488 00000100:00100000:0.0:1789762917.369325:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015119488 00000100:00100000:0.0:1789762917.369363:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440658273502 time 440658368144 state 0x0 ref 1 00000100:00100000:0.0:1789762917.369374:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440858368144 time 440658368144 state 0x1 ref 1 00000001:00080000:2.0:1789762917.506628:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967434 is committed 00000100:00100000:2.0:1789762917.554583:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440858368144 time 440858477534 state 0x0 ref 1 00000100:00100000:2.0:1789762917.554606:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015119488:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762917.554654:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 40894464, mtime: 0x6aad9d65, atime: 0x0, ctime: 0x6aad9d65, blocks: 0x13807 00010000:00010000:2.0:1789762917.554661:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b05654 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.554675:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=440s v=(0 0) 00010000:00010000:2.0:1789762917.554679:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762917.554684:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b05654 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.554695:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b05654 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762917.566677:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968189 is committed 00002000:00010000:2.0:1789762917.566716:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:2.0:1789762917.566720:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 79879 -> 81927 00010000:00010000:2.0:1789762917.566731:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b05654 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.566751:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b05654 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.566757:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b05654 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.566762:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762917.566766:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b05654 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.566773:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b05654 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.566787:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b05654 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.566792:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b05654 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762917.566872:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015119488:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12270us (197603us total) trans 4294968189 rc 0/0 00000100:00100000:2.0:1789762917.566880:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1196 00000100:00100000:2.0:1789762917.566890:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440858368144 time 440858477534 state 0x2 ref 0 00000100:00100000:3.0:1789762917.569713:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762917.569785:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015119616 00000100:00100000:2.0:1789762917.569801:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015119616 00000100:00100000:2.0:1789762917.569838:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 440858368144 time 440858477534 state 0x0 ref 1 00000100:00100000:2.0:1789762917.569850:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441058477534 time 440858477534 state 0x1 ref 1 00000100:00100000:2.0:1789762917.754639:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441058477534 time 441058532970 state 0x0 ref 1 00000100:00100000:2.0:1789762917.754665:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015119616:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762917.754716:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 41943040, mtime: 0x6aad9d65, atime: 0x0, ctime: 0x6aad9d65, blocks: 0x14007 00010000:00010000:2.0:1789762917.754723:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b0565b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.754736:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=441s v=(0 0) 00010000:00010000:2.0:1789762917.754740:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762917.754745:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b0565b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.754757:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b0565b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762917.766596:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968190 is committed 00002000:00010000:2.0:1789762917.766613:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 41943040 -> 42991616 00002000:00010000:2.0:1789762917.766620:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 81927 -> 83975 00010000:00010000:2.0:1789762917.766631:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b0565b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.766641:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b0565b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.766654:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b0565b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.766660:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762917.766664:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b0565b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.766670:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b0565b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.766685:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b0565b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762917.766691:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000227325bf/0x416193e012b0565b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762917.766751:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015119616:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12096us (197037us total) trans 4294968190 rc 0/0 00000100:00100000:2.0:1789762917.766759:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1197 00000100:00100000:2.0:1789762917.766769:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441058477534 time 441058532970 state 0x2 ref 0 00000100:00100000:3.0:1789762917.768956:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762917.768990:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015119744 00000100:00100000:2.0:1789762917.768998:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015119744 00000100:00100000:2.0:1789762917.769034:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441058477534 time 441058532970 state 0x0 ref 1 00000100:00100000:2.0:1789762917.769059:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441258532970 time 441058532970 state 0x1 ref 1 00000100:00100000:1.0:1789762917.954743:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441258532970 time 441258624664 state 0x0 ref 1 00000100:00100000:1.0:1789762917.954780:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015119744:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762917.954852:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 42991616, mtime: 0x6aad9d65, atime: 0x0, ctime: 0x6aad9d65, blocks: 0x14807 00010000:00010000:1.0:1789762917.954863:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005922f58c/0x416193e012b05662 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.954880:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=441s v=(0 0) 00010000:00010000:1.0:1789762917.954885:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762917.954892:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005922f58c/0x416193e012b05662 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.954907:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005922f58c/0x416193e012b05662 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762917.967109:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968191 is committed 00002000:00010000:1.0:1789762917.967112:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 42991616 -> 44040192 00002000:00010000:1.0:1789762917.967118:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 83975 -> 86023 00010000:00010000:1.0:1789762917.967128:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005922f58c/0x416193e012b05662 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.967139:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005922f58c/0x416193e012b05662 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.967156:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005922f58c/0x416193e012b05662 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.967161:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4c000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762917.967165:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005922f58c/0x416193e012b05662 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.967172:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005922f58c/0x416193e012b05662 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.967186:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005922f58c/0x416193e012b05662 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762917.967191:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005922f58c/0x416193e012b05662 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762917.967263:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015119744:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12483us (198294us total) trans 4294968191 rc 0/0 00000100:00100000:1.0:1789762917.967272:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1198 00000100:00100000:1.0:1789762917.967282:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441258532970 time 441258624664 state 0x2 ref 0 00000100:00100000:0.0:1789762917.970083:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762917.970138:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015119872 00000100:00100000:1.0:1789762917.970160:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015119872 00000100:00100000:1.0:1789762917.970208:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441258532970 time 441258624664 state 0x0 ref 1 00000100:00100000:1.0:1789762917.970221:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441458624664 time 441258624664 state 0x1 ref 1 00000100:00100000:2.0:1789762918.154813:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441458624664 time 441458708322 state 0x0 ref 1 00000100:00100000:2.0:1789762918.154840:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb24380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015119872:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762918.154891:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 44040192, mtime: 0x6aad9d65, atime: 0x0, ctime: 0x6aad9d65, blocks: 0x15007 00010000:00010000:2.0:1789762918.154899:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05669 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.154920:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=441s v=(0 0) 00010000:00010000:2.0:1789762918.154924:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762918.154929:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05669 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.154941:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05669 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762918.167682:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968192 is committed 00002000:00010000:2.0:1789762918.167720:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 44040192 -> 45088768 00002000:00010000:2.0:1789762918.167725:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762917 -> 1789762918 00002000:00010000:2.0:1789762918.167728:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762917 -> 1789762918 00002000:00010000:2.0:1789762918.167729:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 86023 -> 88071 00010000:00010000:2.0:1789762918.167739:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05669 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.167760:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05669 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.167767:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05669 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.167772:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762918.167775:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05669 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.167782:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05669 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.167797:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05669 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.167802:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05669 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762918.167881:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb24380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015119872:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13049us (197800us total) trans 4294968192 rc 0/0 00000100:00100000:2.0:1789762918.167890:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1199 00000100:00100000:2.0:1789762918.167899:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441458624664 time 441458708322 state 0x2 ref 0 00000100:00100000:0.0:1789762918.170734:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762918.170779:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015120000 00000100:00100000:2.0:1789762918.170792:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015120000 00000100:00100000:2.0:1789762918.170830:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441458624664 time 441458708322 state 0x0 ref 1 00000100:00100000:2.0:1789762918.170841:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441658708322 time 441458708322 state 0x1 ref 1 00010000:00010000:3.0:1789762918.327270:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762918.354905:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441658708322 time 441658800252 state 0x0 ref 1 00000100:00100000:2.0:1789762918.354931:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015120000:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762918.354980:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 45088768, mtime: 0x6aad9d66, atime: 0x0, ctime: 0x6aad9d66, blocks: 0x15807 00010000:00010000:2.0:1789762918.354987:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05670 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.355001:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=441s v=(0 0) 00010000:00010000:2.0:1789762918.355005:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762918.355009:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05670 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.355020:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05670 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762918.368832:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968193 is committed 00002000:00010000:0.0:1789762918.368842:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 45088768 -> 46137344 00002000:00010000:0.0:1789762918.368849:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 88071 -> 90121 00010000:00010000:0.0:1789762918.368861:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05670 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.368871:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05670 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.368886:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05670 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.368891:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762918.368895:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05670 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.368901:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05670 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.368916:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05670 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.368922:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05670 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762918.368993:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb27b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015120000:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14070us (198256us total) trans 4294968193 rc 0/0 00000100:00100000:0.0:1789762918.369001:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1200 00000100:00100000:0.0:1789762918.369011:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441658708322 time 441658800252 state 0x2 ref 0 00000100:00100000:3.0:1789762918.372209:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762918.372305:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015120128 00000100:00100000:2.0:1789762918.372326:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015120128 00000100:00100000:2.0:1789762918.372366:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441658708322 time 441658800252 state 0x0 ref 1 00000100:00100000:2.0:1789762918.372378:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441858800252 time 441658800252 state 0x1 ref 1 00000004:00080000:0.0:1789762918.528986:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 30 committed 00000100:00100000:2.0:1789762918.529140:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d82f92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022191360:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762918.529201:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762918.529250:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022191360 00000100:00100000:1.0:1789762918.529263:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022191360 00000100:00080000:2.0:1789762918.529309:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762918.529374:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:1.0:1789762918.529397:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022191360:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1789762918.529444:0:6647:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:22 count 1 00010000:00010000:1.0:1789762918.529505:0:6647:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762918.529553:0:6647:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b053d0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd800 expref: 6 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762918.529562:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=9s v=(0 0) 00010000:00010000:1.0:1789762918.529567:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762918.529572:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762918.529606:0:6647:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b053d0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd800 expref: 6 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762918.529622:0:6647:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b053d0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd800 expref: 6 pid: 6647 timeout: 541 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762918.529637:0:6647:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc435180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6647:1876702022191488:192.168.203.48@tcp:104: 00000100:00100000:1.0:1789762918.529732:0:6647:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e878bac00 going to sleep for 11 seconds 00000100:00100000:0.0:1789762918.531120:0:6647:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc435180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6647:1876702022191488:192.168.203.48@tcp:104: 00010000:00010000:0.0:1789762918.531144:0:6647:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762918.531254:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762918.531293:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015120256 00000100:00100000:0.0:1789762918.531300:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015120256 00010000:00010000:0.0:1789762918.531327:0:5790:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b053d0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd800 expref: 7 pid: 6647 timeout: 541 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762918.531360:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 64 00000100:00100000:0.0:1789762918.531368:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2364:x1876702015120256:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762918.531379:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762918.531385:0:5790:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b053d0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd800 expref: 7 pid: 6647 timeout: 541 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.531399:0:5790:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b053d0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd800 expref: 7 pid: 6647 timeout: 541 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.531425:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b053d0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400010020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd800 expref: 7 pid: 6647 timeout: 541 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.531438:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762918.531442:0:5790:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762918.531447:0:5790:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.531456:0:5790:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.531512:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762918.531541:0:6647:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762918.531573:0:6647:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762918.531588:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015120256:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 214us (329us total) trans 0 rc 0/0 00010000:00010000:1.0:1789762918.531597:0:6647:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762918.531625:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 64 00010000:00010000:1.0:1789762918.531629:0:6647:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762918.531660:0:6647:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762918.531706:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762918.531720:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762918.531734:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4dc00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762918.531740:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762918.531800:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762918.531817:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762918.531825:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042596384/0x416193e012b05677 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762918.531847:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762918.531849:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762918.531851:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762918.532446:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022191360:12345-0@lo:6:osp-syn-0-0.0 Request processed in 3055us (3240us total) trans 4294968194 rc 0/0 00000100:00100000:1.0:1789762918.532456:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000004:00080000:2.0:1789762918.532530:0:3272:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981d82f92a00/2, rc 0, transno 4294968194 00000100:00100000:2.0:1789762918.532537:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d82f92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022191360:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789762918.555047:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441858800252 time 441858937773 state 0x0 ref 1 00000100:00100000:1.0:1789762918.555073:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015120128:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762918.555149:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 46137344, mtime: 0x6aad9d66, atime: 0x0, ctime: 0x6aad9d66, blocks: 0x16009 00010000:00010000:1.0:1789762918.555158:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045bc480f/0x416193e012b0567e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762918.555173:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=441s v=(0 0) 00010000:00010000:1.0:1789762918.555177:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762918.555183:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000045bc480f/0x416193e012b0567e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762918.555195:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000045bc480f/0x416193e012b0567e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762918.573918:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 46137344 -> 47185920 00002000:00010000:0.0:1789762918.573928:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 90121 -> 92169 00010000:00010000:0.0:1789762918.573947:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000045bc480f/0x416193e012b0567e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762918.573959:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968195 is committed 00010000:00010000:0.0:1789762918.573965:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000045bc480f/0x416193e012b0567e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.573980:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000045bc480f/0x416193e012b0567e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762918.573999:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968194 is committed 00010000:00010000:0.0:1789762918.574013:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762918.574018:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000045bc480f/0x416193e012b0567e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.574029:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000045bc480f/0x416193e012b0567e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.574049:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000045bc480f/0x416193e012b0567e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.574075:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000045bc480f/0x416193e012b0567e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762918.574232:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015120128:12345-192.168.203.48@tcp:4:dd.500 Request processed in 19161us (202021us total) trans 4294968195 rc 0/0 00000100:00100000:0.0:1789762918.574246:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1201 00000100:00100000:0.0:1789762918.574261:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441858800252 time 441858937773 state 0x2 ref 0 00000100:00100000:3.0:1789762918.577468:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762918.577539:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015120384 00000100:00100000:1.0:1789762918.577550:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015120384 00000100:00100000:1.0:1789762918.577604:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 441858800252 time 441858937773 state 0x0 ref 1 00000100:00100000:1.0:1789762918.577620:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442058937773 time 441858937773 state 0x1 ref 1 00000100:00100000:2.0:1789762918.755199:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442058937773 time 442059093240 state 0x0 ref 1 00000100:00100000:2.0:1789762918.755226:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82086d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015120384:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762918.755278:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 47185920, mtime: 0x6aad9d66, atime: 0x0, ctime: 0x6aad9d66, blocks: 0x16809 00010000:00010000:2.0:1789762918.755285:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05685 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.755299:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=442s v=(0 0) 00010000:00010000:2.0:1789762918.755302:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762918.755307:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05685 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762918.755320:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05685 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762918.769119:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968196 is committed 00002000:00010000:0.0:1789762918.769134:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 47185920 -> 48234496 00002000:00010000:0.0:1789762918.769139:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 92169 -> 94217 00010000:00010000:0.0:1789762918.769154:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05685 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.769165:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05685 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.769172:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05685 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.769178:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762918.769182:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05685 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.769190:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05685 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.769222:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05685 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.769228:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f309428d/0x416193e012b05685 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762918.769311:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82086d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015120384:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14093us (191842us total) trans 4294968196 rc 0/0 00000100:00100000:0.0:1789762918.769321:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1202 00000100:00100000:0.0:1789762918.769332:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442058937773 time 442059093240 state 0x2 ref 0 00000100:00100000:2.0:1789762918.772269:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762918.772322:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015120512 00000100:00100000:3.0:1789762918.772333:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015120512 00000100:00100000:3.0:1789762918.772387:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442058937773 time 442059093240 state 0x0 ref 1 00000100:00100000:3.0:1789762918.772401:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442259093240 time 442059093240 state 0x1 ref 1 00000100:00100000:0.0:1789762918.955334:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442259093240 time 442259224107 state 0x0 ref 1 00000100:00100000:0.0:1789762918.955370:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015120512:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762918.955439:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 48234496, mtime: 0x6aad9d66, atime: 0x0, ctime: 0x6aad9d66, blocks: 0x17009 00010000:00010000:0.0:1789762918.955449:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0568c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.955466:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=442s v=(0 0) 00010000:00010000:0.0:1789762918.955470:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762918.955488:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0568c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.955505:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0568c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762918.967888:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968197 is committed 00002000:00010000:0.0:1789762918.967912:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 48234496 -> 49283072 00002000:00010000:0.0:1789762918.967918:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 94217 -> 96265 00010000:00010000:0.0:1789762918.967933:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0568c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.967949:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0568c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.967962:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0568c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.967970:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762918.967974:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0568c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.967983:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0568c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.968002:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0568c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762918.968007:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0568c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762918.968127:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015120512:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12769us (195854us total) trans 4294968197 rc 0/0 00000100:00100000:0.0:1789762918.968146:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1203 00000100:00100000:0.0:1789762918.968158:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442259093240 time 442259224107 state 0x2 ref 0 00000100:00100000:3.0:1789762918.970383:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762918.970429:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015120640 00000100:00100000:2.0:1789762918.970440:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015120640 00000100:00100000:2.0:1789762918.970507:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442259093240 time 442259224107 state 0x0 ref 1 00000100:00100000:2.0:1789762918.970521:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442459224107 time 442259224107 state 0x1 ref 1 00000100:00100000:0.0:1789762919.155466:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442459224107 time 442459355969 state 0x0 ref 1 00000100:00100000:0.0:1789762919.155540:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015120640:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:2.0:1789762919.155566:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789762919.155614:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 49283072, mtime: 0x6aad9d66, atime: 0x0, ctime: 0x6aad9d66, blocks: 0x17809 00010000:00010000:0.0:1789762919.155623:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05693 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.155640:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=442s v=(0 0) 00010000:00010000:0.0:1789762919.155644:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762919.155650:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05693 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.155663:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05693 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762919.168997:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 49283072 -> 50331648 00000001:00080000:3.0:1789762919.168997:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968198 is committed 00002000:00010000:0.0:1789762919.169008:0:14912:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762918 -> 1789762919 00002000:00010000:0.0:1789762919.169011:0:14912:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762918 -> 1789762919 00002000:00010000:0.0:1789762919.169014:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 96265 -> 98313 00010000:00010000:0.0:1789762919.169029:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05693 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.169052:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05693 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.169059:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05693 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.169064:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762919.169069:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05693 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.169077:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05693 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.169112:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05693 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.169118:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05693 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762919.169187:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015120640:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13684us (198804us total) trans 4294968198 rc 0/0 00000100:00100000:0.0:1789762919.169197:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1204 00000100:00100000:0.0:1789762919.169206:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442459224107 time 442459355969 state 0x2 ref 0 00000100:00100000:2.0:1789762919.171540:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762919.171591:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015120768 00000100:00100000:1.0:1789762919.171602:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015120768 00000100:00100000:1.0:1789762919.171654:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442459224107 time 442459355969 state 0x0 ref 1 00000100:00100000:1.0:1789762919.171666:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442659355969 time 442459355969 state 0x1 ref 1 00010000:00010000:0.0:1789762919.351234:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762919.355563:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442659355969 time 442659453959 state 0x0 ref 1 00000100:00100000:0.0:1789762919.355595:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015120768:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762919.355664:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 50331648, mtime: 0x6aad9d67, atime: 0x0, ctime: 0x6aad9d67, blocks: 0x18009 00010000:00010000:0.0:1789762919.355673:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0569a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.355690:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=442s v=(0 0) 00010000:00010000:0.0:1789762919.355694:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762919.355701:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0569a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.355713:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0569a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762919.369219:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 50331648 -> 51380224 00000001:00080000:2.0:1789762919.369219:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968199 is committed 00002000:00010000:0.0:1789762919.369230:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 98313 -> 100361 00010000:00010000:0.0:1789762919.369243:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0569a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.369253:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0569a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.369261:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0569a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.369277:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762919.369282:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0569a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.369289:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0569a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.369307:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0569a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.369312:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0569a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762919.369382:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015120768:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13799us (197841us total) trans 4294968199 rc 0/0 00000100:00100000:0.0:1789762919.369391:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1205 00000100:00100000:0.0:1789762919.369401:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442659355969 time 442659453959 state 0x2 ref 0 00000100:00100000:3.0:1789762919.371655:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762919.371705:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015120896 00000100:00100000:2.0:1789762919.371715:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015120896 00000100:00100000:2.0:1789762919.371776:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442659355969 time 442659453959 state 0x0 ref 1 00000100:00100000:2.0:1789762919.371793:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442859453959 time 442659453959 state 0x1 ref 1 00000100:00100000:0.0:1789762919.555693:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442859453959 time 442859579097 state 0x0 ref 1 00000100:00100000:0.0:1789762919.555736:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82086680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015120896:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762919.555809:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 51380224, mtime: 0x6aad9d67, atime: 0x0, ctime: 0x6aad9d67, blocks: 0x18809 00010000:00010000:0.0:1789762919.555818:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.555853:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=442s v=(0 0) 00010000:00010000:0.0:1789762919.555857:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762919.555863:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.555877:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762919.569571:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968200 is committed 00002000:00010000:0.0:1789762919.569608:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 51380224 -> 52428800 00002000:00010000:0.0:1789762919.569614:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 100361 -> 102409 00010000:00010000:0.0:1789762919.569631:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.569643:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.569651:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.569657:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762919.569662:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.569670:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.569701:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.569707:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762919.569809:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82086680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015120896:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14090us (198150us total) trans 4294968200 rc 0/0 00000100:00100000:0.0:1789762919.569819:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1206 00000100:00100000:0.0:1789762919.569832:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442859453959 time 442859579097 state 0x2 ref 0 00000100:00100000:3.0:1789762919.572699:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762919.572787:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015121024 00000100:00100000:2.0:1789762919.572799:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015121024 00000100:00100000:2.0:1789762919.572853:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 442859453959 time 442859579097 state 0x0 ref 1 00000100:00100000:2.0:1789762919.572866:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443059579097 time 442859579097 state 0x1 ref 1 00000100:00100000:0.0:1789762919.755781:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443059579097 time 443059669652 state 0x0 ref 1 00000100:00100000:0.0:1789762919.755816:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82084700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015121024:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762919.755887:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 52428800, mtime: 0x6aad9d67, atime: 0x0, ctime: 0x6aad9d67, blocks: 0x19009 00010000:00010000:0.0:1789762919.755896:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.755914:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=443s v=(0 0) 00010000:00010000:0.0:1789762919.755918:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762919.755924:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.755941:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762919.769560:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968201 is committed 00002000:00010000:0.0:1789762919.769639:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 52428800 -> 53477376 00002000:00010000:0.0:1789762919.769644:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 102409 -> 104457 00010000:00010000:0.0:1789762919.769658:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.769668:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.769676:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.769684:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762919.769690:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.769701:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.769726:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.769735:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056a8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762919.769819:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82084700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015121024:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14016us (197124us total) trans 4294968201 rc 0/0 00000100:00100000:0.0:1789762919.769836:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1207 00000100:00100000:0.0:1789762919.769846:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443059579097 time 443059669652 state 0x2 ref 0 00000100:00100000:3.0:1789762919.772322:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762919.772410:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015121152 00000100:00100000:1.0:1789762919.772420:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015121152 00000100:00100000:1.0:1789762919.772482:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443059579097 time 443059669652 state 0x0 ref 1 00000100:00100000:1.0:1789762919.772496:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443259669652 time 443059669652 state 0x1 ref 1 00000100:00100000:0.0:1789762919.955883:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443259669652 time 443259773425 state 0x0 ref 1 00000100:00100000:0.0:1789762919.955917:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015121152:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762919.955986:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 53477376, mtime: 0x6aad9d67, atime: 0x0, ctime: 0x6aad9d67, blocks: 0x19809 00010000:00010000:0.0:1789762919.955995:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056af lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.956011:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=443s v=(0 0) 00010000:00010000:0.0:1789762919.956016:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762919.956029:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.956044:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762919.969926:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968202 is committed 00002000:00010000:0.0:1789762919.969988:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:0.0:1789762919.969993:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 104457 -> 106505 00010000:00010000:0.0:1789762919.970005:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056af lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.970016:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056af lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.970031:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.970037:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762919.970041:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056af lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.970049:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056af lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.970067:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056af lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762919.970073:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056af lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762919.970184:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015121152:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14280us (197866us total) trans 4294968202 rc 0/0 00000100:00100000:0.0:1789762919.970194:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1208 00000100:00100000:0.0:1789762919.970205:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443259669652 time 443259773425 state 0x2 ref 0 00000100:00100000:3.0:1789762919.972762:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762919.972817:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015121280 00000100:00100000:2.0:1789762919.972827:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015121280 00000100:00100000:2.0:1789762919.972873:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443259669652 time 443259773425 state 0x0 ref 1 00000100:00100000:2.0:1789762919.972887:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443459773425 time 443259773425 state 0x1 ref 1 00000100:00100000:0.0:1789762920.155966:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443459773425 time 443459852812 state 0x0 ref 1 00000100:00100000:0.0:1789762920.156000:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82087480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015121280:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762920.156113:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 54525952, mtime: 0x6aad9d67, atime: 0x0, ctime: 0x6aad9d67, blocks: 0x1a009 00010000:00010000:0.0:1789762920.156125:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056b6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.156156:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=443s v=(0 0) 00010000:00010000:0.0:1789762920.156163:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762920.156172:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.156191:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762920.169006:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968203 is committed 00002000:00010000:0.0:1789762920.169039:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 54525952 -> 55574528 00002000:00010000:0.0:1789762920.169045:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762919 -> 1789762920 00002000:00010000:0.0:1789762920.169048:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762919 -> 1789762920 00002000:00010000:0.0:1789762920.169051:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 106505 -> 108553 00010000:00010000:0.0:1789762920.169065:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056b6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.169079:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056b6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.169110:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.169121:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762920.169136:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056b6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.169147:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056b6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.169171:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056b6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.169181:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056b6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762920.169278:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82087480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015121280:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13285us (196511us total) trans 4294968203 rc 0/0 00000100:00100000:0.0:1789762920.169291:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1209 00000100:00100000:0.0:1789762920.169305:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443459773425 time 443459852812 state 0x2 ref 0 00000100:00100000:3.0:1789762920.171592:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762920.171636:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015121408 00000100:00100000:2.0:1789762920.171647:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015121408 00000100:00100000:2.0:1789762920.171695:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443459773425 time 443459852812 state 0x0 ref 1 00000100:00100000:2.0:1789762920.171708:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443659852812 time 443459852812 state 0x1 ref 1 00000100:00100000:0.0:1789762920.356069:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443659852812 time 443659955149 state 0x0 ref 1 00000100:00100000:0.0:1789762920.356136:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82086300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015121408:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762920.356209:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 55574528, mtime: 0x6aad9d68, atime: 0x0, ctime: 0x6aad9d68, blocks: 0x1a809 00010000:00010000:0.0:1789762920.356219:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056bd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.356239:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=443s v=(0 0) 00010000:00010000:0.0:1789762920.356257:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762920.356266:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.356283:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762920.371041:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968204 is committed 00002000:00010000:0.0:1789762920.371111:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 55574528 -> 56623104 00002000:00010000:0.0:1789762920.371118:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 108553 -> 110601 00010000:00010000:0.0:1789762920.371137:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056bd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.371154:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056bd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.371165:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.371175:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762920.371181:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056bd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.371192:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056bd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.371219:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056bd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.371243:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056bd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762920.371387:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82086300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015121408:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15259us (199787us total) trans 4294968204 rc 0/0 00000100:00100000:0.0:1789762920.371402:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1210 00000100:00100000:0.0:1789762920.371435:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443659852812 time 443659955149 state 0x2 ref 0 00000100:00100000:2.0:1789762920.374165:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762920.374220:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015121536 00000100:00100000:1.0:1789762920.374232:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015121536 00000100:00100000:1.0:1789762920.374306:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443659852812 time 443659955149 state 0x0 ref 1 00000100:00100000:1.0:1789762920.374320:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443859955149 time 443659955149 state 0x1 ref 1 00010000:00010000:0.0:1789762920.375185:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762920.546334:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762920.546368:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762920.546381:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015121664 00000100:00100000:3.0:1789762920.546391:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015121664 00000100:00100000:0.0:1789762920.546399:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762920.546437:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762920.546451:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 346 00000100:00100000:3.0:1789762920.546479:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015121664:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762920.546502:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015121792 00000100:00100000:0.0:1789762920.546507:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015122048 00000100:00100000:0.0:1789762920.546515:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015122048 00000100:00100000:1.0:1789762920.546521:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015121792 00000100:00100000:2.0:1789762920.546530:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015121920 00000100:00100000:2.0:1789762920.546543:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015121920 00000100:00100000:3.0:1789762920.546569:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015121664:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 107us (240us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762920.546583:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 346 00000100:00100000:0.0:1789762920.546601:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 352 00000100:00100000:0.0:1789762920.546615:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb26a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015122048:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762920.546627:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 353 00000100:00100000:2.0:1789762920.546654:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 176 00000100:00100000:1.0:1789762920.546656:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702015121792:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762920.546669:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb26a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015122048:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 53us (225us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762920.546683:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015121920:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762920.546690:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 352 00000100:00080000:0.0:1789762920.546726:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762920.546736:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762920.546754:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb26d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702015121792:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 110us (380us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762920.546767:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015121920:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 95us (362us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762920.546772:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 353 00000100:00100000:2.0:1789762920.546791:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 176 00000100:00100000:0.0:1789762920.556143:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443859955149 time 443860043710 state 0x0 ref 1 00000100:00100000:0.0:1789762920.556160:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac24a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015121536:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762920.556213:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 56623104, mtime: 0x6aad9d68, atime: 0x0, ctime: 0x6aad9d68, blocks: 0x1b009 00010000:00010000:0.0:1789762920.556221:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056c4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.556234:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=443s v=(0 0) 00010000:00010000:0.0:1789762920.556237:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762920.556242:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.556254:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762920.567317:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc437100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022191616:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762920.567405:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762920.567544:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022191616 00000100:00100000:2.0:1789762920.567553:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022191616 00000100:00080000:3.0:1789762920.567618:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762920.567661:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 147 00000100:00100000:2.0:1789762920.567680:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82084a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022191616:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762920.567728:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 528416768 pending 1703936 free 3807379456 avail 3801088000 00000100:00100000:2.0:1789762920.567822:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82084a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022191616:12345-0@lo:13:osp-pre-0-0.0 Request processed in 148us (413us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762920.567836:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 147 00000100:00100000:2.0:1789762920.567914:0:3273:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294968194 gen 2 00000004:00080000:2.0:1789762920.567920:0:3273:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981d82f92a00, transno 4294968194 00000100:00100000:2.0:1789762920.567949:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc437100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022191616:0@lo:13:osp-pre-0-0.0 00002000:00010000:0.0:1789762920.574054:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:0.0:1789762920.574059:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 110601 -> 112649 00000001:00080000:3.0:1789762920.574059:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968205 is committed 00010000:00010000:0.0:1789762920.574074:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056c4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.574083:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056c4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.574132:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.574139:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762920.574143:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056c4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.574161:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056c4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.574178:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056c4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.574184:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056c4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762920.574246:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac24a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015121536:12345-192.168.203.48@tcp:4:dd.500 Request processed in 18088us (200081us total) trans 4294968205 rc 0/0 00000100:00100000:0.0:1789762920.574255:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1211 00000100:00100000:0.0:1789762920.574265:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443859955149 time 443860043710 state 0x2 ref 0 00000100:00100000:0.0:1789762920.577109:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762920.577170:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015122176 00000100:00100000:2.0:1789762920.577195:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015122176 00000100:00100000:2.0:1789762920.577244:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 443859955149 time 443860043710 state 0x0 ref 1 00000100:00100000:2.0:1789762920.577259:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444060043710 time 443860043710 state 0x1 ref 1 00000100:00100000:0.0:1789762920.756247:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444060043710 time 444060142687 state 0x0 ref 1 00000100:00100000:0.0:1789762920.756273:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015122176:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762920.756325:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 57671680, mtime: 0x6aad9d68, atime: 0x0, ctime: 0x6aad9d68, blocks: 0x1b809 00010000:00010000:0.0:1789762920.756332:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056cb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.756345:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=444s v=(0 0) 00010000:00010000:0.0:1789762920.756349:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762920.756367:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.756380:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762920.771066:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968206 is committed 00002000:00010000:0.0:1789762920.771080:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:0.0:1789762920.771105:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 112649 -> 114697 00010000:00010000:0.0:1789762920.771117:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056cb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.771126:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056cb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.771133:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.771138:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762920.771142:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056cb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.771149:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056cb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.771165:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056cb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.771181:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056cb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762920.771256:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015122176:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14990us (194163us total) trans 4294968206 rc 0/0 00000100:00100000:0.0:1789762920.771268:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1212 00000100:00100000:0.0:1789762920.771278:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444060043710 time 444060142687 state 0x2 ref 0 00000100:00100000:3.0:1789762920.773556:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762920.773609:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015122304 00000100:00100000:2.0:1789762920.773622:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015122304 00000100:00100000:2.0:1789762920.773682:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444060043710 time 444060142687 state 0x0 ref 1 00000100:00100000:2.0:1789762920.773696:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444260142687 time 444060142687 state 0x1 ref 1 00000100:00100000:0.0:1789762920.956452:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444260142687 time 444260334885 state 0x0 ref 1 00000100:00100000:0.0:1789762920.956524:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015122304:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762920.956620:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 58720256, mtime: 0x6aad9d68, atime: 0x0, ctime: 0x6aad9d68, blocks: 0x1c009 00010000:00010000:0.0:1789762920.956632:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.956655:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=444s v=(0 0) 00010000:00010000:0.0:1789762920.956661:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762920.956669:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.956690:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762920.972249:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968207 is committed 00002000:00010000:0.0:1789762920.972272:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:0.0:1789762920.972287:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 114697 -> 116745 00010000:00010000:0.0:1789762920.972299:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.972309:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.972316:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.972321:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762920.972325:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.972331:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.972347:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762920.972352:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762920.972436:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015122304:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15931us (198876us total) trans 4294968207 rc 0/0 00000100:00100000:0.0:1789762920.972449:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1213 00000100:00100000:0.0:1789762920.972477:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444260142687 time 444260334885 state 0x2 ref 0 00000100:00100000:3.0:1789762920.974802:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762920.974852:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015122432 00000100:00100000:1.0:1789762920.974864:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015122432 00000100:00100000:1.0:1789762920.974917:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444260142687 time 444260334885 state 0x0 ref 1 00000100:00100000:1.0:1789762920.974931:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444460334885 time 444260334885 state 0x1 ref 1 00000100:00100000:0.0:1789762921.079266:0:3272:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789762921.079277:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789762921.079289:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac27800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022192000:0@lo:400:kworker.0 00000100:00100000:3.0:1789762921.079301:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4bcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022192128:0@lo:400:kworker.0 00000100:00100000:1.0:1789762921.079329:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022192384:0@lo:400:kworker.0 00000100:00100000:2.0:1789762921.079381:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022192256:0@lo:400:kworker.0 00000100:00100000:0.0:1789762921.079383:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762921.079437:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022192000 00000100:00100000:3.0:1789762921.079448:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762921.079449:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022192000 00000100:00100000:2.0:1789762921.079476:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762921.079507:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022191872:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762921.079541:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789762921.079556:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022192384 00000100:00100000:3.0:1789762921.079556:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022192512:0@lo:400:kworker.0 00000100:00100000:1.0:1789762921.079565:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022192384 00000100:00100000:0.0:1789762921.079571:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762921.079580:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:2.0:1789762921.079598:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb27100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022192000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762921.079632:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762921.079638:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:1.0:1789762921.079651:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82084380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022192384:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762921.079686:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022191872 00000100:00100000:0.0:1789762921.079694:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022191872 00000100:00100000:2.0:1789762921.079696:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb27100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022192000:12345-0@lo:400:kworker.0 Request processed in 103us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762921.079706:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022192256 00000100:00100000:2.0:1789762921.079707:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00100000:3.0:1789762921.079716:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022192256 00000100:00100000:1.0:1789762921.079717:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82084380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022192384:12345-0@lo:400:kworker.0 Request processed in 68us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762921.079724:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00100000:2.0:1789762921.079744:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762921.079772:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac26680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022191744:0@lo:400:kworker.0 00000100:00100000:1.0:1789762921.079777:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022192128 00000100:00100000:3.0:1789762921.079778:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022192512 00000100:00100000:3.0:1789762921.079785:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022192512 00000100:00100000:1.0:1789762921.079786:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022192128 00000100:00100000:0.0:1789762921.079794:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:0.0:1789762921.079816:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb24e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022191872:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762921.079819:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762921.079827:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:1.0:1789762921.079839:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:3.0:1789762921.079846:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022192256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762921.079859:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022192128:12345-0@lo:400:kworker.0 00002000:00000024:0.0:1789762921.079874:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762921.079874:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022191744 00000100:00100000:2.0:1789762921.079886:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022191744 00000100:00100000:3.0:1789762921.079906:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022192256:12345-0@lo:400:kworker.0 Request processed in 62us (442us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762921.079915:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:2.0:1789762921.079926:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 347 00000100:00100000:3.0:1789762921.079936:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:2.0:1789762921.079938:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850d8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022191744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762921.079942:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022192128:12345-0@lo:400:kworker.0 Request processed in 91us (197us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762921.079954:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:3.0:1789762921.079955:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82087b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022192512:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762921.079971:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb24e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022191872:12345-0@lo:13:osp-pre-1-0.0 Request processed in 158us (405us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762921.079984:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00100000:1.0:1789762921.080006:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4bcdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022192128:0@lo:400:kworker.0 00000100:00100000:3.0:1789762921.080032:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82087b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022192512:12345-0@lo:400:kworker.0 Request processed in 81us (400us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762921.080038:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac26680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022191744:0@lo:400:kworker.0 00000100:00100000:2.0:1789762921.080043:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850d8050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022191744:12345-0@lo:400:kworker.0 Request processed in 104us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762921.080047:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:2.0:1789762921.080058:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 347 00000100:00100000:0.0:1789762921.080080:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac27800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022192000:0@lo:400:kworker.0 00000100:00080000:3.0:1789762921.080082:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789762921.080133:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762921.080169:0:3272:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967434 gen 2 00000004:00080000:0.0:1789762921.080176:0:3272:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebac25180, transno 4294967433 00000100:00100000:3.0:1789762921.080183:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022192256:0@lo:400:kworker.0 00000004:00080000:0.0:1789762921.080193:0:3272:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebac26300, transno 4294967434 00000100:00100000:2.0:1789762921.080205:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022192512:0@lo:400:kworker.0 00000100:00100000:0.0:1789762921.080216:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022191872:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762921.080237:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022192384:0@lo:400:kworker.0 00000100:00100000:0.0:1789762921.156574:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444460334885 time 444460469437 state 0x0 ref 1 00000100:00100000:0.0:1789762921.156611:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015122432:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:3.0:1789762921.156647:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789762921.156667:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 59768832, mtime: 0x6aad9d68, atime: 0x0, ctime: 0x6aad9d68, blocks: 0x1c809 00010000:00010000:0.0:1789762921.156675:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.156701:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=444s v=(0 0) 00010000:00010000:0.0:1789762921.156705:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762921.156710:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.156721:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762921.170347:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 59768832 -> 60817408 00002000:00010000:0.0:1789762921.170353:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762920 -> 1789762921 00002000:00010000:0.0:1789762921.170356:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762920 -> 1789762921 00002000:00010000:0.0:1789762921.170358:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 116745 -> 118793 00010000:00010000:0.0:1789762921.170370:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762921.170371:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968208 is committed 00010000:00010000:0.0:1789762921.170382:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.170390:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.170395:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762921.170399:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.170416:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.170432:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.170437:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762921.170532:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015122432:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13939us (195730us total) trans 4294968208 rc 0/0 00000100:00100000:0.0:1789762921.170545:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1214 00000100:00100000:0.0:1789762921.170555:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444460334885 time 444460469437 state 0x2 ref 0 00000100:00100000:2.0:1789762921.172793:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762921.172840:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015122560 00000100:00100000:3.0:1789762921.172849:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015122560 00000100:00100000:3.0:1789762921.172889:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444460334885 time 444460469437 state 0x0 ref 1 00000100:00100000:3.0:1789762921.172900:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444660469437 time 444460469437 state 0x1 ref 1 00000100:00100000:0.0:1789762921.356656:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444660469437 time 444660551569 state 0x0 ref 1 00000100:00100000:0.0:1789762921.356681:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015122560:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762921.356731:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 60817408, mtime: 0x6aad9d69, atime: 0x0, ctime: 0x6aad9d69, blocks: 0x1d009 00010000:00010000:0.0:1789762921.356739:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.356751:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=444s v=(0 0) 00010000:00010000:0.0:1789762921.356755:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762921.356760:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.356791:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762921.371004:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 60817408 -> 61865984 00002000:00010000:0.0:1789762921.371010:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 118793 -> 120841 00000001:00080000:3.0:1789762921.371016:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968209 is committed 00010000:00010000:0.0:1789762921.371022:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.371036:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.371048:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.371056:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762921.371061:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.371071:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.371106:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.371112:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762921.371189:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015122560:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14516us (198398us total) trans 4294968209 rc 0/0 00000100:00100000:0.0:1789762921.371197:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1215 00000100:00100000:0.0:1789762921.371207:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444660469437 time 444660551569 state 0x2 ref 0 00000100:00100000:2.0:1789762921.373602:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762921.373634:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015122688 00000100:00100000:0.0:1789762921.373642:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015122688 00000100:00100000:0.0:1789762921.373681:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444660469437 time 444660551569 state 0x0 ref 1 00000100:00100000:0.0:1789762921.373692:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444860551569 time 444660551569 state 0x1 ref 1 00010000:00010000:3.0:1789762921.399189:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762921.556727:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444860551569 time 444860621414 state 0x0 ref 1 00000100:00100000:0.0:1789762921.556753:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015122688:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762921.556806:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 61865984, mtime: 0x6aad9d69, atime: 0x0, ctime: 0x6aad9d69, blocks: 0x1d809 00010000:00010000:0.0:1789762921.556813:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.556827:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=444s v=(0 0) 00010000:00010000:0.0:1789762921.556830:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762921.556835:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.556846:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762921.567903:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968210 is committed 00002000:00010000:0.0:1789762921.567976:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 61865984 -> 62914560 00002000:00010000:0.0:1789762921.567981:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 120841 -> 122889 00010000:00010000:0.0:1789762921.567992:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.568003:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.568009:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.568014:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762921.568017:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.568024:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.568040:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.568044:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762921.568139:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015122688:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11393us (194534us total) trans 4294968210 rc 0/0 00000100:00100000:0.0:1789762921.568155:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1216 00000100:00100000:0.0:1789762921.568164:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444860551569 time 444860621414 state 0x2 ref 0 00000100:00100000:2.0:1789762921.571265:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762921.571354:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015122816 00000100:00100000:1.0:1789762921.571363:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015122816 00000100:00100000:1.0:1789762921.571438:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 444860551569 time 444860621414 state 0x0 ref 1 00000100:00100000:1.0:1789762921.571489:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445060621414 time 444860621414 state 0x1 ref 1 00000100:00100000:0.0:1789762921.756858:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445060621414 time 445060742669 state 0x0 ref 1 00000100:00100000:0.0:1789762921.756885:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015122816:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762921.756941:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 62914560, mtime: 0x6aad9d69, atime: 0x0, ctime: 0x6aad9d69, blocks: 0x1e009 00010000:00010000:0.0:1789762921.756952:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056ee lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.756969:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=445s v=(0 0) 00010000:00010000:0.0:1789762921.756974:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762921.756981:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.757000:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762921.769292:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968211 is committed 00002000:00010000:0.0:1789762921.769310:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:0.0:1789762921.769315:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 122889 -> 124937 00010000:00010000:0.0:1789762921.769326:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056ee lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.769337:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056ee lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.769350:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.769367:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762921.769371:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056ee lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.769377:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056ee lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.769392:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056ee lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.769398:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056ee lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762921.769481:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015122816:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12605us (198218us total) trans 4294968211 rc 0/0 00000100:00100000:0.0:1789762921.769489:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1217 00000100:00100000:0.0:1789762921.769499:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445060621414 time 445060742669 state 0x2 ref 0 00000100:00100000:3.0:1789762921.772050:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762921.772143:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015122944 00000100:00100000:2.0:1789762921.772151:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015122944 00000100:00100000:2.0:1789762921.772191:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445060621414 time 445060742669 state 0x0 ref 1 00000100:00100000:2.0:1789762921.772213:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445260742669 time 445060742669 state 0x1 ref 1 00000100:00100000:0.0:1789762921.956964:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445260742669 time 445260847224 state 0x0 ref 1 00000100:00100000:0.0:1789762921.956990:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82084e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015122944:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762921.957056:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 63963136, mtime: 0x6aad9d69, atime: 0x0, ctime: 0x6aad9d69, blocks: 0x1e809 00010000:00010000:0.0:1789762921.957063:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056f5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.957076:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=445s v=(0 0) 00010000:00010000:0.0:1789762921.957079:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762921.957084:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.957136:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762921.969706:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968212 is committed 00002000:00010000:0.0:1789762921.969715:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 63963136 -> 65011712 00002000:00010000:0.0:1789762921.969722:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 124937 -> 126985 00010000:00010000:0.0:1789762921.969735:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056f5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.969750:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056f5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.969761:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.969769:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762921.969775:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056f5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.969799:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056f5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.969819:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056f5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762921.969829:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056f5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762921.969899:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82084e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015122944:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12915us (197845us total) trans 4294968212 rc 0/0 00000100:00100000:0.0:1789762921.969909:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1218 00000100:00100000:0.0:1789762921.969921:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445260742669 time 445260847224 state 0x2 ref 0 00000100:00100000:2.0:1789762921.973375:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762921.973414:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015123072 00000100:00100000:0.0:1789762921.973423:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015123072 00000100:00100000:0.0:1789762921.973485:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445260742669 time 445260847224 state 0x0 ref 1 00000100:00100000:0.0:1789762921.973498:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445460847224 time 445260847224 state 0x1 ref 1 00000100:00100000:0.0:1789762922.157039:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445460847224 time 445460933260 state 0x0 ref 1 00000100:00100000:0.0:1789762922.157065:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015123072:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762922.157139:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 65011712, mtime: 0x6aad9d69, atime: 0x0, ctime: 0x6aad9d69, blocks: 0x1f009 00010000:00010000:0.0:1789762922.157159:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056fc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.157173:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=445s v=(0 0) 00010000:00010000:0.0:1789762922.157176:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762922.157181:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.157193:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762922.170567:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 65011712 -> 66060288 00000001:00080000:3.0:1789762922.170567:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968213 is committed 00002000:00010000:0.0:1789762922.170574:0:14912:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762921 -> 1789762922 00002000:00010000:0.0:1789762922.170577:0:14912:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762921 -> 1789762922 00002000:00010000:0.0:1789762922.170579:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 126985 -> 129033 00010000:00010000:0.0:1789762922.170591:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056fc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.170601:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056fc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.170608:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.170613:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762922.170617:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056fc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.170634:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056fc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.170649:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056fc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.170654:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b056fc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762922.170723:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015123072:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13666us (197346us total) trans 4294968213 rc 0/0 00000100:00100000:0.0:1789762922.170732:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1219 00000100:00100000:0.0:1789762922.170742:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445460847224 time 445460933260 state 0x2 ref 0 00000100:00100000:2.0:1789762922.174542:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762922.174589:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015123200 00000100:00100000:1.0:1789762922.174598:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015123200 00000100:00100000:1.0:1789762922.174650:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445460847224 time 445460933260 state 0x0 ref 1 00000100:00100000:1.0:1789762922.174663:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445660933260 time 445460933260 state 0x1 ref 1 00000100:00100000:0.0:1789762922.357158:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445660933260 time 445661048267 state 0x0 ref 1 00000100:00100000:0.0:1789762922.357186:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015123200:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762922.357239:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 66060288, mtime: 0x6aad9d6a, atime: 0x0, ctime: 0x6aad9d6a, blocks: 0x1f809 00010000:00010000:0.0:1789762922.357246:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05703 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.357260:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=445s v=(0 0) 00010000:00010000:0.0:1789762922.357263:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762922.357268:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05703 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.357294:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05703 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762922.370496:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968214 is committed 00002000:00010000:0.0:1789762922.370508:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 66060288 -> 67108864 00002000:00010000:0.0:1789762922.370515:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 129033 -> 131081 00010000:00010000:0.0:1789762922.370529:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05703 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.370544:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05703 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.370552:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05703 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.370557:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762922.370561:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05703 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.370568:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05703 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.370583:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05703 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.370589:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05703 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762922.370666:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015123200:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13488us (196122us total) trans 4294968214 rc 0/0 00000100:00100000:0.0:1789762922.370676:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1220 00000100:00100000:0.0:1789762922.370686:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445660933260 time 445661048267 state 0x2 ref 0 00000100:00100000:2.0:1789762922.374301:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762922.374341:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015123328 00000100:00100000:3.0:1789762922.374349:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015123328 00000100:00100000:3.0:1789762922.374389:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445660933260 time 445661048267 state 0x0 ref 1 00000100:00100000:3.0:1789762922.374400:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445861048267 time 445661048267 state 0x1 ref 1 00010000:00010000:3.0:1789762922.423209:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762922.557247:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445861048267 time 445861141760 state 0x0 ref 1 00000100:00100000:0.0:1789762922.557272:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015123328:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762922.557323:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 67108864, mtime: 0x6aad9d6a, atime: 0x0, ctime: 0x6aad9d6a, blocks: 0x20009 00010000:00010000:0.0:1789762922.557330:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0570a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.557344:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=445s v=(0 0) 00010000:00010000:0.0:1789762922.557347:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762922.557352:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0570a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.557364:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0570a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762922.569833:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968215 is committed 00002000:00010000:0.0:1789762922.569842:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 67108864 -> 68157440 00002000:00010000:0.0:1789762922.569849:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 131081 -> 133129 00010000:00010000:0.0:1789762922.569862:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0570a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.569883:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0570a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.569890:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0570a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.569895:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762922.569899:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0570a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.569905:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0570a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.569922:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0570a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.569930:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0570a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762922.570003:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015123328:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12739us (195701us total) trans 4294968215 rc 0/0 00000100:00100000:0.0:1789762922.570012:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1221 00000100:00100000:0.0:1789762922.570022:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445861048267 time 445861141760 state 0x2 ref 0 00000100:00100000:3.0:1789762922.572419:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762922.572463:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015123456 00000100:00100000:0.0:1789762922.572477:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015123456 00000100:00100000:0.0:1789762922.572529:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 445861048267 time 445861141760 state 0x0 ref 1 00000100:00100000:0.0:1789762922.572545:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446061141760 time 445861141760 state 0x1 ref 1 00000100:00100000:0.0:1789762922.757316:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446061141760 time 446061210852 state 0x0 ref 1 00000100:00100000:0.0:1789762922.757341:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82086a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015123456:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762922.757392:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 68157440, mtime: 0x6aad9d6a, atime: 0x0, ctime: 0x6aad9d6a, blocks: 0x20809 00010000:00010000:0.0:1789762922.757399:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05711 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.757413:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=446s v=(0 0) 00010000:00010000:0.0:1789762922.757417:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762922.757422:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05711 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.757434:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05711 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762922.768871:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968216 is committed 00002000:00010000:0.0:1789762922.768947:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 68157440 -> 69206016 00002000:00010000:0.0:1789762922.768952:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 133129 -> 135177 00010000:00010000:0.0:1789762922.768962:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05711 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.768972:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05711 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.768987:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05711 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.768992:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762922.768996:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05711 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.769002:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05711 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.769016:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05711 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.769022:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05711 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762922.769117:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82086a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015123456:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11755us (196668us total) trans 4294968216 rc 0/0 00000100:00100000:0.0:1789762922.769128:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1222 00000100:00100000:0.0:1789762922.769140:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446061141760 time 446061210852 state 0x2 ref 0 00000100:00100000:3.0:1789762922.771279:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762922.771332:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015123584 00000100:00100000:1.0:1789762922.771340:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015123584 00000100:00100000:1.0:1789762922.771380:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446061141760 time 446061210852 state 0x0 ref 1 00000100:00100000:1.0:1789762922.771391:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446261210852 time 446061210852 state 0x1 ref 1 00000100:00100000:0.0:1789762922.957416:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446261210852 time 446261313389 state 0x0 ref 1 00000100:00100000:0.0:1789762922.957442:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82087800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015123584:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762922.957519:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 69206016, mtime: 0x6aad9d6a, atime: 0x0, ctime: 0x6aad9d6a, blocks: 0x21009 00010000:00010000:0.0:1789762922.957526:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05718 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.957541:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=446s v=(0 0) 00010000:00010000:0.0:1789762922.957544:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762922.957549:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05718 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.957559:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05718 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762922.968899:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968217 is committed 00002000:00010000:0.0:1789762922.968908:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 69206016 -> 70254592 00002000:00010000:0.0:1789762922.968915:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 135177 -> 137225 00010000:00010000:0.0:1789762922.968929:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05718 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.968944:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05718 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.968955:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05718 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.968963:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762922.968968:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05718 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.968987:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05718 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.969002:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05718 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762922.969008:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05718 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762922.969072:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82087800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015123584:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11638us (197791us total) trans 4294968217 rc 0/0 00000100:00100000:0.0:1789762922.969080:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1223 00000100:00100000:0.0:1789762922.969125:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446261210852 time 446261313389 state 0x2 ref 0 00000100:00100000:3.0:1789762922.971162:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762922.971221:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015123712 00000100:00100000:2.0:1789762922.971231:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015123712 00000100:00100000:2.0:1789762922.971271:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446261210852 time 446261313389 state 0x0 ref 1 00000100:00100000:2.0:1789762922.971282:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446461313389 time 446261313389 state 0x1 ref 1 00000100:00100000:0.0:1789762923.157527:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446461313389 time 446461417760 state 0x0 ref 1 00000100:00100000:0.0:1789762923.157568:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82085500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015123712:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:3.0:1789762923.157604:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789762923.157620:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 70254592, mtime: 0x6aad9d6a, atime: 0x0, ctime: 0x6aad9d6a, blocks: 0x21809 00010000:00010000:0.0:1789762923.157628:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0571f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.157642:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=446s v=(0 0) 00010000:00010000:0.0:1789762923.157646:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762923.157650:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0571f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.157675:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0571f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762923.169129:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968218 is committed 00002000:00010000:0.0:1789762923.169211:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 70254592 -> 71303168 00002000:00010000:0.0:1789762923.169216:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762922 -> 1789762923 00002000:00010000:0.0:1789762923.169219:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762922 -> 1789762923 00002000:00010000:0.0:1789762923.169222:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 137225 -> 139273 00010000:00010000:0.0:1789762923.169235:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0571f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.169249:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0571f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.169259:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0571f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.169268:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762923.169274:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0571f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.169285:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0571f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.169305:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0571f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.169329:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0571f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762923.169411:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82085500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015123712:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11863us (198245us total) trans 4294968218 rc 0/0 00000100:00100000:0.0:1789762923.169423:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1224 00000100:00100000:0.0:1789762923.169435:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446461313389 time 446461417760 state 0x2 ref 0 00000100:00100000:3.0:1789762923.171437:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762923.171489:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015123840 00000100:00100000:0.0:1789762923.171498:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015123840 00000100:00100000:0.0:1789762923.171542:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446461313389 time 446461417760 state 0x0 ref 1 00000100:00100000:0.0:1789762923.171558:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446661417760 time 446461417760 state 0x1 ref 1 00000100:00100000:0.0:1789762923.357619:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446661417760 time 446661509824 state 0x0 ref 1 00000100:00100000:0.0:1789762923.357651:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015123840:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762923.357709:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 71303168, mtime: 0x6aad9d6b, atime: 0x0, ctime: 0x6aad9d6b, blocks: 0x22009 00010000:00010000:0.0:1789762923.357718:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05726 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.357734:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=446s v=(0 0) 00010000:00010000:0.0:1789762923.357738:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762923.357759:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05726 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.357773:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05726 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762923.369900:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968219 is committed 00002000:00010000:0.0:1789762923.369975:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 71303168 -> 72351744 00002000:00010000:0.0:1789762923.369981:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 139273 -> 141321 00010000:00010000:0.0:1789762923.369995:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05726 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.370007:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05726 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.370014:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05726 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.370019:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762923.370023:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05726 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.370029:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05726 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.370044:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05726 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.370059:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05726 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762923.370155:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015123840:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12516us (198717us total) trans 4294968219 rc 0/0 00000100:00100000:0.0:1789762923.370163:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1225 00000100:00100000:0.0:1789762923.370174:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446661417760 time 446661509824 state 0x2 ref 0 00000100:00100000:2.0:1789762923.372265:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762923.372340:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015123968 00000100:00100000:1.0:1789762923.372349:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015123968 00000100:00100000:1.0:1789762923.372390:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446661417760 time 446661509824 state 0x0 ref 1 00000100:00100000:1.0:1789762923.372401:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446861509824 time 446661509824 state 0x1 ref 1 00010000:00010000:0.0:1789762923.447205:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762923.557694:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446861509824 time 446861590177 state 0x0 ref 1 00000100:00100000:0.0:1789762923.557718:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015123968:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762923.557781:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 72351744, mtime: 0x6aad9d6b, atime: 0x0, ctime: 0x6aad9d6b, blocks: 0x22809 00010000:00010000:0.0:1789762923.557791:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0572d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.557808:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=446s v=(0 0) 00010000:00010000:0.0:1789762923.557813:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762923.557825:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0572d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.557838:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0572d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762923.569015:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968220 is committed 00002000:00010000:0.0:1789762923.569026:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 72351744 -> 73400320 00002000:00010000:0.0:1789762923.569032:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 141321 -> 143369 00010000:00010000:0.0:1789762923.569043:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0572d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.569053:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0572d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.569060:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0572d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.569065:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762923.569068:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0572d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.569075:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0572d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.569109:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0572d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.569118:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0572d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762923.569189:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015123968:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11478us (196923us total) trans 4294968220 rc 0/0 00000100:00100000:0.0:1789762923.569205:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1226 00000100:00100000:0.0:1789762923.569215:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446861509824 time 446861590177 state 0x2 ref 0 00000100:00100000:2.0:1789762923.571520:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762923.571560:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015124096 00000100:00100000:3.0:1789762923.571568:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015124096 00000100:00100000:3.0:1789762923.571605:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 446861509824 time 446861590177 state 0x0 ref 1 00000100:00100000:3.0:1789762923.571616:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447061590177 time 446861590177 state 0x1 ref 1 00000100:00100000:0.0:1789762923.757830:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447061590177 time 447061725650 state 0x0 ref 1 00000100:00100000:0.0:1789762923.757856:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015124096:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762923.757907:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 73400320, mtime: 0x6aad9d6b, atime: 0x0, ctime: 0x6aad9d6b, blocks: 0x23009 00010000:00010000:0.0:1789762923.757914:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05734 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.757927:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=447s v=(0 0) 00010000:00010000:0.0:1789762923.757931:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762923.757936:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05734 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.757949:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05734 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762923.769356:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968221 is committed 00002000:00010000:0.0:1789762923.769365:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 73400320 -> 74448896 00002000:00010000:0.0:1789762923.769370:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 143369 -> 145417 00010000:00010000:0.0:1789762923.769381:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05734 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.769400:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05734 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.769420:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05734 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.769425:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762923.769429:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05734 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.769436:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05734 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.769463:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05734 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.769469:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05734 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762923.769531:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015124096:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11684us (198010us total) trans 4294968221 rc 0/0 00000100:00100000:0.0:1789762923.769539:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1227 00000100:00100000:0.0:1789762923.769550:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447061590177 time 447061725650 state 0x2 ref 0 00000100:00100000:3.0:1789762923.771613:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762923.771645:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015124224 00000100:00100000:0.0:1789762923.771653:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015124224 00000100:00100000:0.0:1789762923.771689:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447061590177 time 447061725650 state 0x0 ref 1 00000100:00100000:0.0:1789762923.771709:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447261725650 time 447061725650 state 0x1 ref 1 00000100:00100000:0.0:1789762923.957902:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447261725650 time 447261797959 state 0x0 ref 1 00000100:00100000:0.0:1789762923.957929:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015124224:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762923.957986:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 74448896, mtime: 0x6aad9d6b, atime: 0x0, ctime: 0x6aad9d6b, blocks: 0x23809 00010000:00010000:0.0:1789762923.957997:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0573b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.958015:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=447s v=(0 0) 00010000:00010000:0.0:1789762923.958019:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762923.958023:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0573b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.958035:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0573b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762923.968981:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968222 is committed 00002000:00010000:0.0:1789762923.969047:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:0.0:1789762923.969052:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 145417 -> 147465 00010000:00010000:0.0:1789762923.969061:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0573b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.969070:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0573b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.969113:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0573b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.969122:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762923.969127:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0573b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.969137:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0573b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.969153:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0573b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762923.969159:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0573b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762923.969218:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015124224:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11299us (197604us total) trans 4294968222 rc 0/0 00000100:00100000:0.0:1789762923.969226:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1228 00000100:00100000:0.0:1789762923.969235:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447261725650 time 447261797959 state 0x2 ref 0 00000100:00100000:3.0:1789762923.971129:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762923.971166:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015124352 00000100:00100000:1.0:1789762923.971174:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015124352 00000100:00100000:1.0:1789762923.971215:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447261725650 time 447261797959 state 0x0 ref 1 00000100:00100000:1.0:1789762923.971227:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447461797959 time 447261797959 state 0x1 ref 1 00000100:00100000:0.0:1789762924.158021:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447461797959 time 447461917204 state 0x0 ref 1 00000100:00100000:0.0:1789762924.158047:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015124352:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762924.158142:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 75497472, mtime: 0x6aad9d6b, atime: 0x0, ctime: 0x6aad9d6b, blocks: 0x24009 00010000:00010000:0.0:1789762924.158150:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05742 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.158163:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=447s v=(0 0) 00010000:00010000:0.0:1789762924.158166:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762924.158171:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05742 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.158182:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05742 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789762924.170306:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968223 is committed 00002000:00010000:0.0:1789762924.170319:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:0.0:1789762924.170325:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762923 -> 1789762924 00002000:00010000:0.0:1789762924.170327:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762923 -> 1789762924 00002000:00010000:0.0:1789762924.170329:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 147465 -> 149513 00010000:00010000:0.0:1789762924.170340:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05742 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.170350:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05742 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.170356:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05742 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.170361:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762924.170372:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05742 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.170379:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05742 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.170395:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05742 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.170401:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05742 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762924.170485:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015124352:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12446us (199355us total) trans 4294968223 rc 0/0 00000100:00100000:0.0:1789762924.170494:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1229 00000100:00100000:0.0:1789762924.170504:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447461797959 time 447461917204 state 0x2 ref 0 00000100:00100000:3.0:1789762924.172742:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762924.172789:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015124480 00000100:00100000:2.0:1789762924.172797:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015124480 00000100:00100000:2.0:1789762924.172836:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447461797959 time 447461917204 state 0x0 ref 1 00000100:00100000:2.0:1789762924.172848:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447661917204 time 447461917204 state 0x1 ref 1 00000100:00100000:0.0:1789762924.358149:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447661917204 time 447662045382 state 0x0 ref 1 00000100:00100000:0.0:1789762924.358176:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015124480:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762924.358226:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 76546048, mtime: 0x6aad9d6c, atime: 0x0, ctime: 0x6aad9d6c, blocks: 0x24809 00010000:00010000:0.0:1789762924.358234:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05749 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.358248:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=447s v=(0 0) 00010000:00010000:0.0:1789762924.358264:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762924.358270:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05749 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.358281:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05749 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762924.369513:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 76546048 -> 77594624 00000001:00080000:3.0:1789762924.369515:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968224 is committed 00002000:00010000:0.0:1789762924.369523:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 149513 -> 151561 00010000:00010000:0.0:1789762924.369536:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05749 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.369549:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05749 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.369559:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05749 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.369566:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762924.369570:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05749 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.369577:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05749 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.369592:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05749 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.369609:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05749 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762924.369680:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015124480:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11513us (196937us total) trans 4294968224 rc 0/0 00000100:00100000:0.0:1789762924.369689:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1230 00000100:00100000:0.0:1789762924.369698:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447661917204 time 447662045382 state 0x2 ref 0 00000100:00100000:3.0:1789762924.372016:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762924.372085:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015124608 00000100:00100000:0.0:1789762924.372111:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015124608 00000100:00100000:0.0:1789762924.372150:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447661917204 time 447662045382 state 0x0 ref 1 00000100:00100000:0.0:1789762924.372161:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447862045382 time 447662045382 state 0x1 ref 1 00010000:00010000:0.0:1789762924.471165:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762924.558216:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447862045382 time 447862106012 state 0x0 ref 1 00000100:00100000:0.0:1789762924.558246:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015124608:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762924.558309:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 77594624, mtime: 0x6aad9d6c, atime: 0x0, ctime: 0x6aad9d6c, blocks: 0x25009 00010000:00010000:0.0:1789762924.558321:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05750 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.558339:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=447s v=(0 0) 00010000:00010000:0.0:1789762924.558352:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762924.558360:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05750 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.558376:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05750 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762924.570864:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968225 is committed 00002000:00010000:0.0:1789762924.570934:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:0.0:1789762924.570941:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 151561 -> 153609 00010000:00010000:0.0:1789762924.570956:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05750 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.570970:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05750 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.570980:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05750 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.570988:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762924.570991:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05750 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.570998:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05750 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.571014:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05750 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.571030:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05750 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762924.571126:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015124608:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12889us (199108us total) trans 4294968225 rc 0/0 00000100:00100000:0.0:1789762924.571135:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1231 00000100:00100000:0.0:1789762924.571145:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447862045382 time 447862106012 state 0x2 ref 0 00000100:00100000:3.0:1789762924.573539:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762924.573591:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015124736 00000100:00100000:1.0:1789762924.573601:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015124736 00000100:00100000:1.0:1789762924.573646:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 447862045382 time 447862106012 state 0x0 ref 1 00000100:00100000:1.0:1789762924.573662:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448062106012 time 447862106012 state 0x1 ref 1 00000100:00100000:0.0:1789762924.758323:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448062106012 time 448062219024 state 0x0 ref 1 00000100:00100000:0.0:1789762924.758349:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015124736:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762924.758401:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 78643200, mtime: 0x6aad9d6c, atime: 0x0, ctime: 0x6aad9d6c, blocks: 0x25809 00010000:00010000:0.0:1789762924.758408:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05757 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.758421:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=448s v=(0 0) 00010000:00010000:0.0:1789762924.758425:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762924.758429:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05757 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.758457:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05757 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762924.770234:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968226 is committed 00002000:00010000:0.0:1789762924.770246:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:0.0:1789762924.770257:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 153609 -> 155657 00010000:00010000:0.0:1789762924.770268:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05757 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.770278:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05757 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.770285:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05757 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.770289:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762924.770293:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05757 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.770300:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05757 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.770315:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05757 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.770320:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05757 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762924.770383:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc868380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015124736:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12043us (196842us total) trans 4294968226 rc 0/0 00000100:00100000:0.0:1789762924.770392:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1232 00000100:00100000:0.0:1789762924.770401:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448062106012 time 448062219024 state 0x2 ref 0 00000100:00100000:3.0:1789762924.772724:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762924.772775:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015124864 00000100:00100000:2.0:1789762924.772783:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015124864 00000100:00100000:2.0:1789762924.772834:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448062106012 time 448062219024 state 0x0 ref 1 00000100:00100000:2.0:1789762924.772846:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448262219024 time 448062219024 state 0x1 ref 1 00000100:00100000:0.0:1789762924.958536:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448262219024 time 448262427677 state 0x0 ref 1 00000100:00100000:0.0:1789762924.958565:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015124864:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762924.958637:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 79691776, mtime: 0x6aad9d6c, atime: 0x0, ctime: 0x6aad9d6c, blocks: 0x26009 00010000:00010000:0.0:1789762924.958650:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0575e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.958672:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=448s v=(0 0) 00010000:00010000:0.0:1789762924.958678:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762924.958687:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0575e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.958705:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0575e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762924.975192:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968227 is committed 00002000:00010000:0.0:1789762924.975275:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:0.0:1789762924.975283:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 155657 -> 157705 00010000:00010000:0.0:1789762924.975314:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0575e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.975327:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0575e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.975334:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0575e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.975351:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762924.975372:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0575e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.975380:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0575e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.975396:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0575e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762924.975402:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0575e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762924.975514:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc868a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015124864:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16955us (202785us total) trans 4294968227 rc 0/0 00000100:00100000:0.0:1789762924.975525:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1233 00000100:00100000:0.0:1789762924.975538:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448262219024 time 448262427677 state 0x2 ref 0 00000100:00100000:2.0:1789762924.979409:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762924.979488:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015124992 00000100:00100000:0.0:1789762924.979497:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015124992 00000100:00100000:0.0:1789762924.979536:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448262219024 time 448262427677 state 0x0 ref 1 00000100:00100000:0.0:1789762924.979547:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448462427677 time 448262427677 state 0x1 ref 1 00000100:00100000:0.0:1789762925.158588:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448462427677 time 448462483577 state 0x0 ref 1 00000100:00100000:0.0:1789762925.158637:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015124992:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:3.0:1789762925.158674:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789762925.158687:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 80740352, mtime: 0x6aad9d6c, atime: 0x0, ctime: 0x6aad9d6c, blocks: 0x26809 00010000:00010000:0.0:1789762925.158695:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05765 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.158710:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=448s v=(0 0) 00010000:00010000:0.0:1789762925.158713:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762925.158718:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05765 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.158736:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05765 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762925.173312:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 80740352 -> 81788928 00002000:00010000:0.0:1789762925.173318:0:14912:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762924 -> 1789762925 00002000:00010000:0.0:1789762925.173321:0:14912:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762924 -> 1789762925 00002000:00010000:0.0:1789762925.173323:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 157705 -> 159753 00010000:00010000:0.0:1789762925.173333:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05765 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762925.173336:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968228 is committed 00010000:00010000:0.0:1789762925.173347:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05765 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.173354:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05765 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.173370:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762925.173374:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05765 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.173381:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05765 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.173396:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05765 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.173401:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05765 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762925.173473:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015124992:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14854us (194065us total) trans 4294968228 rc 0/0 00000100:00100000:0.0:1789762925.173482:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1234 00000100:00100000:0.0:1789762925.173491:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448462427677 time 448462483577 state 0x2 ref 0 00000100:00100000:3.0:1789762925.175660:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762925.175706:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015125120 00000100:00100000:1.0:1789762925.175715:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015125120 00000100:00100000:1.0:1789762925.175753:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448462427677 time 448462483577 state 0x0 ref 1 00000100:00100000:1.0:1789762925.175765:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448662483577 time 448462483577 state 0x1 ref 1 00000100:00100000:0.0:1789762925.358692:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448662483577 time 448662587604 state 0x0 ref 1 00000100:00100000:0.0:1789762925.358717:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015125120:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762925.358768:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 81788928, mtime: 0x6aad9d6d, atime: 0x0, ctime: 0x6aad9d6d, blocks: 0x27009 00010000:00010000:0.0:1789762925.358775:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0576c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.358800:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=448s v=(0 0) 00010000:00010000:0.0:1789762925.358804:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762925.358808:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0576c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.358820:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0576c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762925.371062:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:0.0:1789762925.371068:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 159753 -> 161801 00000001:00080000:2.0:1789762925.371071:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968229 is committed 00010000:00010000:0.0:1789762925.371078:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0576c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.371111:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0576c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.371118:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0576c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.371123:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762925.371126:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0576c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.371132:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0576c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.371157:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0576c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.371163:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0576c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762925.371225:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015125120:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12515us (195563us total) trans 4294968229 rc 0/0 00000100:00100000:0.0:1789762925.371234:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1235 00000100:00100000:0.0:1789762925.371243:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448662483577 time 448662587604 state 0x2 ref 0 00000100:00100000:3.0:1789762925.373609:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762925.373661:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015125248 00000100:00100000:2.0:1789762925.373669:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015125248 00000100:00100000:2.0:1789762925.373721:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448662483577 time 448662587604 state 0x0 ref 1 00000100:00100000:2.0:1789762925.373737:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448862587604 time 448662587604 state 0x1 ref 1 00010000:00010000:3.0:1789762925.495214:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762925.558807:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448862587604 time 448862700019 state 0x0 ref 1 00000100:00100000:0.0:1789762925.558839:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015125248:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762925.558894:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 82837504, mtime: 0x6aad9d6d, atime: 0x0, ctime: 0x6aad9d6d, blocks: 0x27809 00010000:00010000:0.0:1789762925.558904:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05773 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.558919:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=448s v=(0 0) 00010000:00010000:0.0:1789762925.558923:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762925.558929:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05773 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.558943:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05773 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762925.571140:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968230 is committed 00002000:00010000:0.0:1789762925.571143:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 82837504 -> 83886080 00002000:00010000:0.0:1789762925.571149:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 161801 -> 163849 00010000:00010000:0.0:1789762925.571161:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05773 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.571172:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05773 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.571179:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05773 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.571184:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762925.571187:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05773 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.571194:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05773 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.571209:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05773 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.571214:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b05773 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762925.571288:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015125248:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12459us (197677us total) trans 4294968230 rc 0/0 00000100:00100000:0.0:1789762925.571303:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1236 00000100:00100000:0.0:1789762925.571313:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448862587604 time 448862700019 state 0x2 ref 0 00000100:00100000:3.0:1789762925.573877:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762925.573922:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015125376 00000100:00100000:0.0:1789762925.573931:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015125376 00000100:00100000:0.0:1789762925.573977:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 448862587604 time 448862700019 state 0x0 ref 1 00000100:00100000:0.0:1789762925.573993:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449062700019 time 448862700019 state 0x1 ref 1 00000100:00100000:1.0:1789762925.666369:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762925.666418:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762925.666446:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015125632 00000100:00100000:2.0:1789762925.666456:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015125632 00000100:00100000:1.0:1789762925.666467:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762925.666475:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015125504 00000100:00100000:3.0:1789762925.666482:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015125504 00000100:00100000:1.0:1789762925.666516:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762925.666535:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015125888 00000100:00100000:2.0:1789762925.666538:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015125888 00000100:00100000:0.0:1789762925.666539:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015125760 00000100:00100000:3.0:1789762925.666545:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 348 00000100:00100000:0.0:1789762925.666548:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015125760 00000100:00100000:1.0:1789762925.666560:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 357 00000100:00100000:3.0:1789762925.666565:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015125504:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762925.666572:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 358 00000100:00100000:1.0:1789762925.666579:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702015125632:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762925.666588:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015125888:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762925.666606:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 179 00000100:00100000:0.0:1789762925.666618:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015125760:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762925.666631:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015125504:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 75us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762925.666639:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702015125632:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 65us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762925.666642:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015125888:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 54us (123us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762925.666647:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 348 00000100:00100000:0.0:1789762925.666656:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015125760:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 39us (188us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762925.666661:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 357 00000100:00100000:0.0:1789762925.666670:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 179 00000100:00100000:2.0:1789762925.666672:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 358 00000100:00080000:0.0:1789762925.666708:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762925.666714:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762925.687247:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789762925.687264:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc437100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022192640:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762925.687309:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762925.687352:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022192640 00000100:00100000:2.0:1789762925.687360:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022192640 00000100:00080000:0.0:1789762925.687403:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762925.687413:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:2.0:1789762925.687424:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022192640:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762925.687455:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526057472 pending 0 free 3786407936 avail 3773825024 00000100:00100000:2.0:1789762925.687509:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022192640:12345-0@lo:13:osp-pre-0-0.0 Request processed in 86us (197us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762925.687518:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00100000:3.0:1789762925.687539:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc437100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022192640:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762925.758902:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449062700019 time 449062796404 state 0x0 ref 1 00000100:00100000:0.0:1789762925.758928:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015125376:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762925.758986:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 83886080, mtime: 0x6aad9d6d, atime: 0x0, ctime: 0x6aad9d6d, blocks: 0x28009 00010000:00010000:0.0:1789762925.758993:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0577a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.759018:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=449s v=(0 0) 00010000:00010000:0.0:1789762925.759022:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762925.759027:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0577a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.759040:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0577a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762925.771986:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968231 is committed 00002000:00010000:3.0:1789762925.772007:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 83886080 -> 84934656 00002000:00010000:3.0:1789762925.772012:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 163849 -> 165897 00010000:00010000:3.0:1789762925.772022:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0577a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762925.772033:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0577a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762925.772040:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0577a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762925.772045:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762925.772050:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0577a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762925.772060:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0577a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762925.772081:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0577a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762925.772108:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004887b6d3/0x416193e012b0577a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762925.772182:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc868700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015125376:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13262us (198303us total) trans 4294968231 rc 0/0 00000100:00100000:3.0:1789762925.772190:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1237 00000100:00100000:3.0:1789762925.772199:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449062700019 time 449062796404 state 0x2 ref 0 00000100:00100000:2.0:1789762925.774533:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762925.774590:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015126016 00000100:00100000:1.0:1789762925.774599:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015126016 00000100:00100000:1.0:1789762925.774645:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449062700019 time 449062796404 state 0x0 ref 1 00000100:00100000:1.0:1789762925.774661:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449262796404 time 449062796404 state 0x1 ref 1 00000100:00100000:0.0:1789762925.959032:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449262796404 time 449262928044 state 0x0 ref 1 00000100:00100000:0.0:1789762925.959059:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015126016:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762925.959140:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 84934656, mtime: 0x6aad9d6d, atime: 0x0, ctime: 0x6aad9d6d, blocks: 0x28809 00010000:00010000:0.0:1789762925.959154:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05781 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.959168:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=449s v=(0 0) 00010000:00010000:0.0:1789762925.959172:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762925.959176:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05781 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.959187:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05781 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762925.971972:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968232 is committed 00002000:00010000:0.0:1789762925.971975:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:0.0:1789762925.971987:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 165897 -> 167945 00010000:00010000:0.0:1789762925.971997:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05781 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.972007:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05781 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.972014:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05781 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.972019:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762925.972022:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05781 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.972030:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05781 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.972044:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05781 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762925.972049:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05781 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762925.972146:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015126016:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13094us (197608us total) trans 4294968232 rc 0/0 00000100:00100000:0.0:1789762925.972159:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1238 00000100:00100000:0.0:1789762925.972181:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449262796404 time 449262928044 state 0x2 ref 0 00000100:00100000:3.0:1789762925.976206:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762925.976247:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015126144 00000100:00100000:0.0:1789762925.976256:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015126144 00000100:00100000:0.0:1789762925.976296:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449262796404 time 449262928044 state 0x0 ref 1 00000100:00100000:0.0:1789762925.976307:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449462928044 time 449262928044 state 0x1 ref 1 00000100:00100000:0.0:1789762926.159118:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449462928044 time 449462996077 state 0x0 ref 1 00000100:00100000:0.0:1789762926.159143:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015126144:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762926.159191:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 85983232, mtime: 0x6aad9d6d, atime: 0x0, ctime: 0x6aad9d6d, blocks: 0x29009 00010000:00010000:0.0:1789762926.159199:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05788 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.159212:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=449s v=(0 0) 00010000:00010000:0.0:1789762926.159216:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762926.159220:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05788 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.159231:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05788 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762926.171752:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968233 is committed 00002000:00010000:0.0:1789762926.171760:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 85983232 -> 87031808 00002000:00010000:0.0:1789762926.171767:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762925 -> 1789762926 00002000:00010000:0.0:1789762926.171770:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762925 -> 1789762926 00002000:00010000:0.0:1789762926.171773:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 167945 -> 169993 00010000:00010000:0.0:1789762926.171795:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05788 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.171807:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05788 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.171817:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05788 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.171825:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762926.171831:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05788 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.171840:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05788 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.171858:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05788 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.171866:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05788 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762926.171943:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015126144:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12806us (195734us total) trans 4294968233 rc 0/0 00000100:00100000:0.0:1789762926.171951:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1239 00000100:00100000:0.0:1789762926.171961:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449462928044 time 449462996077 state 0x2 ref 0 00000100:00100000:1.0:1789762926.174586:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762926.174620:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015126272 00000100:00100000:0.0:1789762926.174627:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015126272 00000100:00100000:0.0:1789762926.174664:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449462928044 time 449462996077 state 0x0 ref 1 00000100:00100000:0.0:1789762926.174675:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449662996077 time 449462996077 state 0x1 ref 1 00000100:00100000:0.0:1789762926.199190:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789762926.199207:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022192768:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762926.199218:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789762926.199233:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022193280:0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.199255:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022192896:0@lo:400:kworker.0 00000100:00100000:0.0:1789762926.199262:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762926.199273:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022193024:0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.199278:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762926.199307:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762926.199323:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8257d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022193152:0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.199330:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762926.199333:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022192768 00000100:00100000:0.0:1789762926.199338:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022192768 00000100:00100000:1.0:1789762926.199350:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d84540380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022193408:0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.199363:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d84542680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022193536:0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.199363:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762926.199394:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:1.0:1789762926.199399:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762926.199401:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762926.199408:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb24a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022192768:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762926.199451:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022193024 00002000:00000024:0.0:1789762926.199454:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762926.199456:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022193024 00000100:00100000:1.0:1789762926.199470:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022193280 00000100:00100000:1.0:1789762926.199476:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022193280 00000100:00080000:2.0:1789762926.199477:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762926.199496:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022193152 00000100:00100000:3.0:1789762926.199500:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022193152 00000100:00100000:0.0:1789762926.199506:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb24a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022192768:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (244us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762926.199512:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022193408 00000100:00100000:1.0:1789762926.199513:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022193536 00000100:00100000:0.0:1789762926.199517:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:2.0:1789762926.199519:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022193408 00000100:00100000:1.0:1789762926.199520:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022193536 00000100:00100000:3.0:1789762926.199538:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:3.0:1789762926.199550:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022193024:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.199560:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:2.0:1789762926.199565:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:0.0:1789762926.199569:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb25180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022192768:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762926.199573:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022193280:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762926.199581:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022193152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.199599:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022193024:12345-0@lo:400:kworker.0 Request processed in 52us (272us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762926.199605:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00100000:0.0:1789762926.199615:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 361 00000100:00100000:0.0:1789762926.199628:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022193408:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.199633:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022193280:12345-0@lo:400:kworker.0 Request processed in 61us (352us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762926.199635:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762926.199640:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc868e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022193152:12345-0@lo:400:kworker.0 Request processed in 62us (274us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762926.199656:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:2.0:1789762926.199656:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:3.0:1789762926.199661:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022193024:0@lo:400:kworker.0 00000100:00100000:0.0:1789762926.199678:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022193408:12345-0@lo:400:kworker.0 Request processed in 53us (286us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762926.199684:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 361 00000100:00100000:1.0:1789762926.199688:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:1.0:1789762926.199698:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84542d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022193536:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.199708:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022192896 00000100:00100000:3.0:1789762926.199712:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022192896 00000100:00100000:2.0:1789762926.199736:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022193280:0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.199748:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 349 00000100:00100000:0.0:1789762926.199751:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d84540380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022193408:0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.199756:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84542d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022193536:12345-0@lo:400:kworker.0 Request processed in 60us (356us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762926.199759:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022193152:0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.199761:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022192896:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762926.199777:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:3.0:1789762926.199817:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022192896:12345-0@lo:400:kworker.0 Request processed in 61us (510us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762926.199821:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d84542680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022193536:0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.199829:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 349 00000100:00100000:0.0:1789762926.199834:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8257e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022192896:0@lo:400:kworker.0 00000100:00100000:3.0:1789762926.359198:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449662996077 time 449663093094 state 0x0 ref 1 00000100:00100000:3.0:1789762926.359223:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015126272:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762926.359274:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 87031808, mtime: 0x6aad9d6e, atime: 0x0, ctime: 0x6aad9d6e, blocks: 0x29809 00010000:00010000:3.0:1789762926.359282:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b0578f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.359295:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=449s v=(0 0) 00010000:00010000:3.0:1789762926.359299:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762926.359303:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b0578f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.359314:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b0578f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762926.370543:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968234 is committed 00002000:00010000:3.0:1789762926.370559:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 87031808 -> 88080384 00002000:00010000:3.0:1789762926.370565:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 169993 -> 172041 00010000:00010000:3.0:1789762926.370575:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b0578f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.370585:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b0578f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.370593:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b0578f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.370598:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762926.370601:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b0578f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.370608:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b0578f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.370621:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b0578f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.370626:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b0578f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762926.370683:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015126272:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11468us (196096us total) trans 4294968234 rc 0/0 00000100:00100000:3.0:1789762926.370691:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1240 00000100:00100000:3.0:1789762926.370702:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449662996077 time 449663093094 state 0x2 ref 0 00000100:00100000:1.0:1789762926.372792:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762926.372845:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015126400 00000100:00100000:0.0:1789762926.372853:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015126400 00000100:00100000:0.0:1789762926.372891:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449662996077 time 449663093094 state 0x0 ref 1 00000100:00100000:0.0:1789762926.372911:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449863093094 time 449663093094 state 0x1 ref 1 00010000:00010000:3.0:1789762926.519230:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762926.559269:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449863093094 time 449863163451 state 0x0 ref 1 00000100:00100000:0.0:1789762926.559296:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015126400:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762926.559348:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 88080384, mtime: 0x6aad9d6e, atime: 0x0, ctime: 0x6aad9d6e, blocks: 0x2a009 00010000:00010000:0.0:1789762926.559355:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05796 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.559368:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=449s v=(0 0) 00010000:00010000:0.0:1789762926.559371:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762926.559376:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05796 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.559387:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05796 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762926.570391:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968235 is committed 00002000:00010000:0.0:1789762926.570451:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 88080384 -> 89128960 00002000:00010000:0.0:1789762926.570456:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 172041 -> 174089 00010000:00010000:0.0:1789762926.570466:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05796 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.570482:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05796 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.570489:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05796 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.570494:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762926.570498:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05796 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.570505:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05796 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.570520:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05796 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.570525:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b05796 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762926.570588:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015126400:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11301us (197794us total) trans 4294968235 rc 0/0 00000100:00100000:0.0:1789762926.570597:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1241 00000100:00100000:0.0:1789762926.570607:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449863093094 time 449863163451 state 0x2 ref 0 00000100:00100000:1.0:1789762926.572825:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762926.572858:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015126528 00000100:00100000:0.0:1789762926.572873:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015126528 00000100:00100000:0.0:1789762926.572916:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 449863093094 time 449863163451 state 0x0 ref 1 00000100:00100000:0.0:1789762926.572928:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450063163451 time 449863163451 state 0x1 ref 1 00000100:00100000:0.0:1789762926.759329:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450063163451 time 450063219964 state 0x0 ref 1 00000100:00100000:0.0:1789762926.759360:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015126528:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762926.759439:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 89128960, mtime: 0x6aad9d6e, atime: 0x0, ctime: 0x6aad9d6e, blocks: 0x2a809 00010000:00010000:0.0:1789762926.759447:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b0579d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.759462:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=450s v=(0 0) 00010000:00010000:0.0:1789762926.759466:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762926.759470:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b0579d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762926.759483:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b0579d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762926.770866:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968236 is committed 00002000:00010000:3.0:1789762926.770930:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 89128960 -> 90177536 00002000:00010000:3.0:1789762926.770937:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 174089 -> 176137 00010000:00010000:3.0:1789762926.770950:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b0579d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.770964:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b0579d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.770975:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b0579d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.770983:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762926.770988:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b0579d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.770999:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b0579d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.771018:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b0579d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.771027:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bc5cda3e/0x416193e012b0579d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762926.771128:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015126528:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11776us (198299us total) trans 4294968236 rc 0/0 00000100:00100000:3.0:1789762926.771137:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1242 00000100:00100000:3.0:1789762926.771147:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450063163451 time 450063219964 state 0x2 ref 0 00000100:00100000:2.0:1789762926.773300:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762926.773377:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015126656 00000100:00100000:0.0:1789762926.773387:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015126656 00000100:00100000:0.0:1789762926.773439:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450063163451 time 450063219964 state 0x0 ref 1 00000100:00100000:0.0:1789762926.773450:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450263219964 time 450063219964 state 0x1 ref 1 00000100:00100000:3.0:1789762926.959441:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450263219964 time 450263324786 state 0x0 ref 1 00000100:00100000:3.0:1789762926.959469:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015126656:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762926.959520:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 90177536, mtime: 0x6aad9d6e, atime: 0x0, ctime: 0x6aad9d6e, blocks: 0x2b009 00010000:00010000:3.0:1789762926.959528:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057a4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.959541:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=450s v=(0 0) 00010000:00010000:3.0:1789762926.959544:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762926.959549:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.959560:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762926.970825:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968237 is committed 00002000:00010000:3.0:1789762926.970842:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 90177536 -> 91226112 00002000:00010000:3.0:1789762926.970849:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 176137 -> 178185 00010000:00010000:3.0:1789762926.970863:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057a4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.970878:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057a4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.970889:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.970898:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762926.970903:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057a4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.970914:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057a4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.970932:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057a4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762926.970941:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057a4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762926.971018:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015126656:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11557us (197715us total) trans 4294968237 rc 0/0 00000100:00100000:3.0:1789762926.971027:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1243 00000100:00100000:3.0:1789762926.971038:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450263219964 time 450263324786 state 0x2 ref 0 00000100:00100000:2.0:1789762926.973265:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762926.973324:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015126784 00000100:00100000:0.0:1789762926.973334:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015126784 00000100:00100000:0.0:1789762926.973380:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450263219964 time 450263324786 state 0x0 ref 1 00000100:00100000:0.0:1789762926.973395:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450463324786 time 450263324786 state 0x1 ref 1 00000100:00100000:0.0:1789762927.159491:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450463324786 time 450463381876 state 0x0 ref 1 00000100:00100000:0.0:1789762927.159533:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84543800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015126784:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:3.0:1789762927.159555:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789762927.159593:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 91226112, mtime: 0x6aad9d6e, atime: 0x0, ctime: 0x6aad9d6e, blocks: 0x2b809 00010000:00010000:0.0:1789762927.159602:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b057ab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762927.159615:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=450s v=(0 0) 00010000:00010000:0.0:1789762927.159618:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762927.159623:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b057ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762927.159635:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b057ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762927.170630:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968238 is committed 00002000:00010000:3.0:1789762927.170647:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 91226112 -> 92274688 00002000:00010000:3.0:1789762927.170652:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762926 -> 1789762927 00002000:00010000:3.0:1789762927.170655:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762926 -> 1789762927 00002000:00010000:3.0:1789762927.170657:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 178185 -> 180233 00010000:00010000:3.0:1789762927.170667:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b057ab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.170677:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b057ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.170688:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b057ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.170693:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfde00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762927.170697:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b057ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.170704:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b057ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.170721:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b057ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.170729:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002b972c92/0x416193e012b057ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762927.170800:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015126784:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11287us (197534us total) trans 4294968238 rc 0/0 00000100:00100000:3.0:1789762927.170809:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1244 00000100:00100000:3.0:1789762927.170819:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450463324786 time 450463381876 state 0x2 ref 0 00000100:00100000:1.0:1789762927.173023:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762927.173074:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015126912 00000100:00100000:0.0:1789762927.173083:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015126912 00000100:00100000:0.0:1789762927.173137:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450463324786 time 450463381876 state 0x0 ref 1 00000100:00100000:0.0:1789762927.173157:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450663381876 time 450463381876 state 0x1 ref 1 00000100:00100000:3.0:1789762927.359599:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450663381876 time 450663494121 state 0x0 ref 1 00000100:00100000:3.0:1789762927.359624:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015126912:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762927.359675:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 92274688, mtime: 0x6aad9d6f, atime: 0x0, ctime: 0x6aad9d6f, blocks: 0x2c009 00010000:00010000:3.0:1789762927.359682:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.359695:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=450s v=(0 0) 00010000:00010000:3.0:1789762927.359699:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762927.359704:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.359715:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762927.371553:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968239 is committed 00002000:00010000:3.0:1789762927.371573:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:3.0:1789762927.371578:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 180233 -> 182281 00010000:00010000:3.0:1789762927.371589:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.371598:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.371605:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.371610:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762927.371614:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.371625:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.371641:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.371646:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762927.371708:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015126912:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12091us (198684us total) trans 4294968239 rc 0/0 00000100:00100000:3.0:1789762927.371716:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1245 00000100:00100000:3.0:1789762927.371725:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450663381876 time 450663494121 state 0x2 ref 0 00000100:00100000:2.0:1789762927.373947:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762927.374026:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015127040 00000100:00100000:0.0:1789762927.374034:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015127040 00000100:00100000:0.0:1789762927.374072:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450663381876 time 450663494121 state 0x0 ref 1 00000100:00100000:0.0:1789762927.374084:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450863494121 time 450663494121 state 0x1 ref 1 00010000:00010000:0.0:1789762927.543242:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762927.559667:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450863494121 time 450863562752 state 0x0 ref 1 00000100:00100000:3.0:1789762927.559691:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015127040:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762927.559741:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 93323264, mtime: 0x6aad9d6f, atime: 0x0, ctime: 0x6aad9d6f, blocks: 0x2c809 00010000:00010000:3.0:1789762927.559748:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.559762:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=450s v=(0 0) 00010000:00010000:3.0:1789762927.559766:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762927.559770:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.559782:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762927.571407:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968240 is committed 00002000:00010000:3.0:1789762927.571464:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:3.0:1789762927.571469:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 182281 -> 184329 00010000:00010000:3.0:1789762927.571479:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.571489:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.571495:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.571500:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762927.571504:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.571510:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.571524:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.571529:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762927.571591:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015127040:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11907us (197643us total) trans 4294968240 rc 0/0 00000100:00100000:3.0:1789762927.571600:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1246 00000100:00100000:3.0:1789762927.571609:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450863494121 time 450863562752 state 0x2 ref 0 00000100:00100000:2.0:1789762927.573643:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762927.573687:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015127168 00000100:00100000:0.0:1789762927.573695:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015127168 00000100:00100000:0.0:1789762927.573732:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 450863494121 time 450863562752 state 0x0 ref 1 00000100:00100000:0.0:1789762927.573743:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451063562752 time 450863562752 state 0x1 ref 1 00000100:00100000:3.0:1789762927.759747:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451063562752 time 451063644222 state 0x0 ref 1 00000100:00100000:3.0:1789762927.759768:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015127168:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762927.759818:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 94371840, mtime: 0x6aad9d6f, atime: 0x0, ctime: 0x6aad9d6f, blocks: 0x2d009 00010000:00010000:3.0:1789762927.759826:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057c0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.759840:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=451s v=(0 0) 00010000:00010000:3.0:1789762927.759844:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762927.759848:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.759861:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762927.771384:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 94371840 -> 95420416 00000001:00080000:2.0:1789762927.771388:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968241 is committed 00002000:00010000:0.0:1789762927.771390:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 184329 -> 186379 00010000:00010000:0.0:1789762927.771403:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057c0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762927.771424:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057c0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762927.771431:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762927.771447:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762927.771451:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057c0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762927.771458:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057c0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762927.771473:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057c0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762927.771478:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000082488b0f/0x416193e012b057c0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762927.771535:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015127168:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11772us (197892us total) trans 4294968241 rc 0/0 00000100:00100000:0.0:1789762927.771544:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1247 00000100:00100000:0.0:1789762927.771552:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451063562752 time 451063644222 state 0x2 ref 0 00000100:00100000:2.0:1789762927.773611:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762927.773645:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015127296 00000100:00100000:0.0:1789762927.773653:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015127296 00000100:00100000:0.0:1789762927.773689:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451063562752 time 451063644222 state 0x0 ref 1 00000100:00100000:0.0:1789762927.773701:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451263644222 time 451063644222 state 0x1 ref 1 00000100:00100000:3.0:1789762927.959848:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451263644222 time 451263741297 state 0x0 ref 1 00000100:00100000:3.0:1789762927.959879:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015127296:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762927.959935:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 95420416, mtime: 0x6aad9d6f, atime: 0x0, ctime: 0x6aad9d6f, blocks: 0x2d80b 00010000:00010000:3.0:1789762927.959943:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057c7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.959956:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=451s v=(0 0) 00010000:00010000:3.0:1789762927.959960:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762927.959964:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.959976:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762927.970926:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968242 is committed 00002000:00010000:3.0:1789762927.970986:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 95420416 -> 96468992 00002000:00010000:3.0:1789762927.970990:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 186379 -> 188427 00010000:00010000:3.0:1789762927.971000:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057c7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.971009:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.971016:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.971021:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762927.971025:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.971036:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.971050:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762927.971055:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762927.971163:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015127296:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11295us (197550us total) trans 4294968242 rc 0/0 00000100:00100000:3.0:1789762927.971176:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1248 00000100:00100000:3.0:1789762927.971188:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451263644222 time 451263741297 state 0x2 ref 0 00000100:00100000:1.0:1789762927.973369:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762927.973430:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015127424 00000100:00100000:0.0:1789762927.973445:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015127424 00000100:00100000:0.0:1789762927.973482:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451263644222 time 451263741297 state 0x0 ref 1 00000100:00100000:0.0:1789762927.973493:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451463741297 time 451263741297 state 0x1 ref 1 00000100:00100000:3.0:1789762928.159966:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451463741297 time 451463857357 state 0x0 ref 1 00000100:00100000:3.0:1789762928.159995:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015127424:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762928.160050:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 96468992, mtime: 0x6aad9d6f, atime: 0x0, ctime: 0x6aad9d6f, blocks: 0x2e00b 00010000:00010000:3.0:1789762928.160059:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.160077:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=451s v=(0 0) 00010000:00010000:3.0:1789762928.160082:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762928.160121:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.160134:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762928.171690:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968243 is committed 00002000:00010000:3.0:1789762928.171713:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 96468992 -> 97517568 00002000:00010000:3.0:1789762928.171718:0:14912:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762927 -> 1789762928 00002000:00010000:3.0:1789762928.171721:0:14912:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762927 -> 1789762928 00002000:00010000:3.0:1789762928.171722:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 188427 -> 190475 00010000:00010000:3.0:1789762928.171732:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.171743:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.171750:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.171755:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762928.171759:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.171766:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.171781:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.171791:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762928.171856:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac26680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015127424:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11869us (198485us total) trans 4294968243 rc 0/0 00000100:00100000:3.0:1789762928.171867:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1249 00000100:00100000:3.0:1789762928.171880:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451463741297 time 451463857357 state 0x2 ref 0 00000100:00100000:2.0:1789762928.174370:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762928.174480:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015127552 00000100:00100000:0.0:1789762928.174489:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015127552 00000100:00100000:0.0:1789762928.174527:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451463741297 time 451463857357 state 0x0 ref 1 00000100:00100000:0.0:1789762928.174539:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451663857357 time 451463857357 state 0x1 ref 1 00000100:00100000:0.0:1789762928.360029:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451663857357 time 451663924801 state 0x0 ref 1 00000100:00100000:0.0:1789762928.360052:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84542300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015127552:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762928.360131:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 97517568, mtime: 0x6aad9d70, atime: 0x0, ctime: 0x6aad9d70, blocks: 0x2e80b 00010000:00010000:0.0:1789762928.360139:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b057d5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762928.360152:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=451s v=(0 0) 00010000:00010000:0.0:1789762928.360155:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762928.360160:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b057d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762928.360171:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b057d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762928.371852:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968244 is committed 00002000:00010000:3.0:1789762928.371905:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 97517568 -> 98566144 00002000:00010000:3.0:1789762928.371911:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 190475 -> 192523 00010000:00010000:3.0:1789762928.371922:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b057d5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.371935:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b057d5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.371945:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b057d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.371954:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfd000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762928.371959:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b057d5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.371970:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b057d5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.371995:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b057d5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.372001:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000030a83c5c/0x416193e012b057d5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762928.372064:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84542300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015127552:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12018us (197693us total) trans 4294968244 rc 0/0 00000100:00100000:3.0:1789762928.372082:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1250 00000100:00100000:3.0:1789762928.372121:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451663857357 time 451663924801 state 0x2 ref 0 00000100:00100000:2.0:1789762928.374237:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762928.374281:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015127680 00000100:00100000:0.0:1789762928.374289:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015127680 00000100:00100000:0.0:1789762928.374325:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451663857357 time 451663924801 state 0x0 ref 1 00000100:00100000:0.0:1789762928.374336:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451863924801 time 451663924801 state 0x1 ref 1 00000100:00100000:3.0:1789762928.560144:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451863924801 time 451864035776 state 0x0 ref 1 00000100:00100000:3.0:1789762928.560179:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015127680:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762928.560243:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 98566144, mtime: 0x6aad9d70, atime: 0x0, ctime: 0x6aad9d70, blocks: 0x2f00b 00010000:00010000:3.0:1789762928.560250:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057dc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.560275:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=451s v=(0 0) 00010000:00010000:3.0:1789762928.560279:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762928.560283:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.560294:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762928.567189:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789762928.571980:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968245 is committed 00002000:00010000:3.0:1789762928.571985:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 98566144 -> 99614720 00002000:00010000:3.0:1789762928.571990:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 192523 -> 194571 00010000:00010000:3.0:1789762928.572000:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057dc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.572011:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057dc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.572017:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.572022:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762928.572026:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057dc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.572032:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057dc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.572045:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057dc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.572050:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057dc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762928.572135:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015127680:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11969us (197895us total) trans 4294968245 rc 0/0 00000100:00100000:3.0:1789762928.572144:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1251 00000100:00100000:3.0:1789762928.572152:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451863924801 time 451864035776 state 0x2 ref 0 00000100:00100000:1.0:1789762928.575695:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762928.575739:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015127808 00000100:00100000:0.0:1789762928.575747:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015127808 00000100:00100000:0.0:1789762928.575787:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 451863924801 time 451864035776 state 0x0 ref 1 00000100:00100000:0.0:1789762928.575797:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452064035776 time 451864035776 state 0x1 ref 1 00000100:00100000:3.0:1789762928.760207:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452064035776 time 452064100304 state 0x0 ref 1 00000100:00100000:3.0:1789762928.760232:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015127808:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762928.760286:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 99614720, mtime: 0x6aad9d70, atime: 0x0, ctime: 0x6aad9d70, blocks: 0x2f80b 00010000:00010000:3.0:1789762928.760294:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057e3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.760308:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=452s v=(0 0) 00010000:00010000:3.0:1789762928.760312:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762928.760317:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.760331:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762928.776648:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968246 is committed 00002000:00010000:3.0:1789762928.776659:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 99614720 -> 100663296 00002000:00010000:3.0:1789762928.776664:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 194571 -> 196619 00010000:00010000:3.0:1789762928.776675:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057e3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.776685:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057e3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.776692:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.776697:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762928.776701:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057e3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.776707:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057e3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.776721:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057e3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.776726:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057e3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762928.776785:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015127808:12345-192.168.203.48@tcp:4:dd.500 Request processed in 16560us (201089us total) trans 4294968246 rc 0/0 00000100:00100000:3.0:1789762928.776793:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1252 00000100:00100000:3.0:1789762928.776802:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452064035776 time 452064100304 state 0x2 ref 0 00000100:00100000:1.0:1789762928.779820:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762928.779870:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015127936 00000100:00100000:0.0:1789762928.779879:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015127936 00000100:00100000:0.0:1789762928.779917:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452064035776 time 452064100304 state 0x0 ref 1 00000100:00100000:0.0:1789762928.779928:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452264100304 time 452064100304 state 0x1 ref 1 00000100:00100000:3.0:1789762928.960295:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452264100304 time 452264190307 state 0x0 ref 1 00000100:00100000:3.0:1789762928.960322:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015127936:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762928.960376:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 100663296, mtime: 0x6aad9d70, atime: 0x0, ctime: 0x6aad9d70, blocks: 0x3000b 00010000:00010000:3.0:1789762928.960383:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.960397:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=452s v=(0 0) 00010000:00010000:3.0:1789762928.960416:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762928.960422:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.960440:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762928.972328:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968247 is committed 00002000:00010000:3.0:1789762928.972345:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 100663296 -> 101711872 00002000:00010000:3.0:1789762928.972353:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 196619 -> 198667 00010000:00010000:3.0:1789762928.972365:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.972379:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.972389:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.972396:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762928.972400:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.972423:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.972438:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762928.972444:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057ea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762928.972507:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015127936:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12193us (192684us total) trans 4294968247 rc 0/0 00000100:00100000:3.0:1789762928.972518:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1253 00000100:00100000:3.0:1789762928.972529:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452264100304 time 452264190307 state 0x2 ref 0 00000100:00100000:1.0:1789762928.975581:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762928.975641:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015128064 00000100:00100000:0.0:1789762928.975652:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015128064 00000100:00100000:0.0:1789762928.975695:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452264100304 time 452264190307 state 0x0 ref 1 00000100:00100000:0.0:1789762928.975706:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452464190307 time 452264190307 state 0x1 ref 1 00000100:00100000:3.0:1789762929.160381:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452464190307 time 452464277429 state 0x0 ref 1 00000100:00100000:3.0:1789762929.160435:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015128064:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:2.0:1789762929.160478:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789762929.160494:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 101711872, mtime: 0x6aad9d70, atime: 0x0, ctime: 0x6aad9d70, blocks: 0x3080b 00010000:00010000:3.0:1789762929.160504:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057f1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.160521:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=452s v=(0 0) 00010000:00010000:3.0:1789762929.160526:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762929.160534:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.160548:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762929.171327:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968248 is committed 00002000:00010000:0.0:1789762929.171329:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:0.0:1789762929.171335:0:6652:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 1789762928 -> 1789762929 00002000:00010000:0.0:1789762929.171337:0:6652:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 1789762928 -> 1789762929 00002000:00010000:0.0:1789762929.171339:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 198667 -> 200715 00010000:00010000:0.0:1789762929.171350:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057f1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.171360:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057f1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.171367:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.171373:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762929.171376:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057f1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.171383:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057f1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.171397:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057f1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.171418:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005dcf6e99/0x416193e012b057f1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762929.171498:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015128064:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11099us (195915us total) trans 4294968248 rc 0/0 00000100:00100000:0.0:1789762929.171507:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1254 00000100:00100000:0.0:1789762929.171516:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452464190307 time 452464277429 state 0x2 ref 0 00000100:00100000:3.0:1789762929.173428:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762929.173461:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015128192 00000100:00100000:0.0:1789762929.173469:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015128192 00000100:00100000:0.0:1789762929.173506:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452464190307 time 452464277429 state 0x0 ref 1 00000100:00100000:0.0:1789762929.173518:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452664277429 time 452464277429 state 0x1 ref 1 00000100:00100000:3.0:1789762929.360516:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452664277429 time 452664411574 state 0x0 ref 1 00000100:00100000:3.0:1789762929.360543:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc869f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015128192:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762929.360594:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 102760448, mtime: 0x6aad9d71, atime: 0x0, ctime: 0x6aad9d71, blocks: 0x3100b 00010000:00010000:3.0:1789762929.360601:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057f8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.360614:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=452s v=(0 0) 00010000:00010000:3.0:1789762929.360618:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762929.360622:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.360633:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762929.372301:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968249 is committed 00002000:00010000:3.0:1789762929.372355:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:3.0:1789762929.372360:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 200715 -> 202763 00010000:00010000:3.0:1789762929.372369:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057f8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.372383:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057f8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.372389:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.372394:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762929.372411:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057f8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.372419:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057f8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.372433:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057f8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.372438:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057f8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762929.372495:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc869f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015128192:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11962us (199067us total) trans 4294968249 rc 0/0 00000100:00100000:3.0:1789762929.372503:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1255 00000100:00100000:3.0:1789762929.372511:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452664277429 time 452664411574 state 0x2 ref 0 00000100:00100000:3.0:1789762929.374656:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762929.374735:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015128320 00000100:00100000:0.0:1789762929.374743:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015128320 00000100:00100000:0.0:1789762929.374780:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452664277429 time 452664411574 state 0x0 ref 1 00000100:00100000:0.0:1789762929.374791:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452864411574 time 452664411574 state 0x1 ref 1 00000100:00100000:3.0:1789762929.560637:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452864411574 time 452864531487 state 0x0 ref 1 00000100:00100000:3.0:1789762929.560663:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc86b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015128320:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762929.560727:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 103809024, mtime: 0x6aad9d71, atime: 0x0, ctime: 0x6aad9d71, blocks: 0x3180b 00010000:00010000:3.0:1789762929.560734:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057ff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.560757:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=452s v=(0 0) 00010000:00010000:3.0:1789762929.560760:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762929.560765:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.560776:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762929.576029:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968250 is committed 00002000:00010000:3.0:1789762929.576037:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 103809024 -> 104857600 00002000:00010000:3.0:1789762929.576043:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 202763 -> 204811 00010000:00010000:3.0:1789762929.576053:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057ff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.576062:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057ff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.576078:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.576105:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762929.576109:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057ff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.576115:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057ff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.576128:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057ff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.576134:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000813c13c4/0x416193e012b057ff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762929.576199:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc86b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015128320:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15545us (201542us total) trans 4294968250 rc 0/0 00000100:00100000:3.0:1789762929.576207:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1256 00000100:00100000:3.0:1789762929.576214:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452864411574 time 452864531487 state 0x2 ref 0 00000100:00100000:1.0:1789762929.578419:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762929.578502:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015128576 00000100:00100000:2.0:1789762929.578511:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015128576 00000100:00080000:3.0:1789762929.578550:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762929.578560:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 114 00000100:00100000:2.0:1789762929.578576:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:15435:x1876702015128576:12345-192.168.203.48@tcp:35:dd.500 00000001:00080000:2.0:1789762929.578759:0:5807:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967606 00000100:00100000:2.0:1789762929.578823:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15435:x1876702015128576:12345-192.168.203.48@tcp:35:dd.500 Request processed in 249us (404us total) trans 4294967607 rc 0/0 00000100:00100000:2.0:1789762929.578828:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 114 00000020:00080000:2.0:1789762929.578832:0:5807:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1789762929.591225:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762929.645779:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762929.645847:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015128832 00000100:00100000:3.0:1789762929.645858:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015128832 00000100:00100000:3.0:1789762929.645924:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 359 00000100:00100000:3.0:1789762929.645933:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15453:x1876702015128832:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762929.645955:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762929.645974:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05806 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd854 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.646068:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6000/0x416193e012b0580d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.646111:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6000/0x416193e012b0580d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.646122:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6000/0x416193e012b0580d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762929.646131:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3c:0x0]: returning size 104857600 00010000:00010000:3.0:1789762929.646245:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6600/0x416193e012b05806 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd854 expref: 13 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.646259:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6000/0x416193e012b0580d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd854 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.646266:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762929.646268:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:3.0:1789762929.646271:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981d865a6000, rc 0) 00000100:00100000:3.0:1789762929.646321:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15453:x1876702015128832:12345-192.168.203.48@tcp:101:dd.500 Request processed in 390us (551us total) trans 4294967608 rc 0/0 00000100:00100000:3.0:1789762929.646326:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 359 00000100:00100000:3.0:1789762929.648075:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762929.648137:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015128960 00000100:00100000:0.0:1789762929.648143:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015128960 00000100:00100000:0.0:1789762929.648191:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 211 00000100:00100000:0.0:1789762929.648198:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015128960:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762929.648207:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762929.648219:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0581b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd85b expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762929.648266:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 104857600, mtime: 0x6aad9d71, atime: 0x0, ctime: 0x6aad9d71, blocks: 0x3200b 00010000:00010000:0.0:1789762929.648279:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=452s v=(0 0) 00010000:00010000:0.0:1789762929.648283:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762929.648290:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0581b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd85b expref: 6 pid: 6881 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.648300:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0581b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd85b expref: 6 pid: 6881 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.648308:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc000, rc 0) 00000100:00100000:0.0:1789762929.648338:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015128960:12345-192.168.203.48@tcp:101:dd.500 Request processed in 141us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762929.648353:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 211 00000100:00100000:1.0:1789762929.653043:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762929.653117:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015129088 00000100:00100000:0.0:1789762929.653122:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015129088 00000100:00100000:0.0:1789762929.653165:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374420567340 time 374420680233 state 0x0 ref 1 00000100:00100000:0.0:1789762929.653172:0:12582:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 9: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981e880d3e00 gen 0 rate 10 depth 3 token 0 deadline 374420567340 time 374420680233 state 0x0 ref 1 00000100:00100000:0.0:1789762929.653191:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 3 deadline 453157098377 time 452957098377 state 0x1 ref 1 00000100:00100000:0.0:1789762929.653213:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 2 deadline 453157098377 time 452957122049 state 0x0 ref 1 00000100:00100000:0.0:1789762929.653231:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015129088:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762929.653249:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b05822 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.653269:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762929.653272:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762929.653275:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b05822 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.653283:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b05822 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.659055:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b05822 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.659076:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b05822 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.659083:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b05822 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.659110:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762929.659114:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b05822 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.659121:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b05822 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.659151:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b05822 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.659157:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001328146e/0x416193e012b05822 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762929.659209:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015129088:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5976us (6164us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762929.659220:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1257 00000100:00100000:0.0:1789762929.659231:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 2 deadline 453157098377 time 452957122049 state 0x2 ref 0 00000100:00100000:2.0:1789762929.661716:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762929.661761:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015129216 00000100:00100000:0.0:1789762929.661767:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015129216 00000100:00100000:0.0:1789762929.661790:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 2 deadline 453157098377 time 452957122049 state 0x0 ref 1 00000100:00100000:0.0:1789762929.661798:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 2 deadline 453157122049 time 452957122049 state 0x1 ref 1 00000100:00100000:0.0:1789762929.661824:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 453157122049 time 452965731241 state 0x0 ref 1 00000100:00100000:0.0:1789762929.661835:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015129216:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762929.661854:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05829 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.661875:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762929.661886:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762929.661889:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05829 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.661897:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05829 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.665123:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05829 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.665132:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05829 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.665138:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05829 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.665144:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762929.665147:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05829 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.665153:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05829 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.665164:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05829 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762929.665169:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05829 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762929.665225:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015129216:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3391us (3509us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762929.665231:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1258 00000100:00100000:0.0:1789762929.665237:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 453157122049 time 452965731241 state 0x2 ref 0 00000100:00100000:3.0:1789762929.667795:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762929.667853:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015129344 00000100:00100000:1.0:1789762929.667861:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015129344 00000100:00100000:1.0:1789762929.667897:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 453157122049 time 452965731241 state 0x0 ref 1 00000100:00100000:1.0:1789762929.667906:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 1 deadline 453165731241 time 452965731241 state 0x1 ref 1 00000100:00100000:1.0:1789762929.667929:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453165731241 time 452971838645 state 0x0 ref 1 00000100:00100000:1.0:1789762929.667940:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015129344:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762929.667962:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003dd12418/0x416193e012b05830 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762929.667974:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762929.667979:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762929.667984:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003dd12418/0x416193e012b05830 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762929.667997:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003dd12418/0x416193e012b05830 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762929.671011:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003dd12418/0x416193e012b05830 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762929.671021:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003dd12418/0x416193e012b05830 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762929.671028:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003dd12418/0x416193e012b05830 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762929.671033:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec04800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762929.671036:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003dd12418/0x416193e012b05830 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762929.671043:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003dd12418/0x416193e012b05830 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762929.671055:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003dd12418/0x416193e012b05830 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762929.671063:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003dd12418/0x416193e012b05830 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762929.671158:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015129344:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3219us (3373us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762929.671165:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1259 00000100:00100000:1.0:1789762929.671171:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453165731241 time 452971838645 state 0x2 ref 0 00000100:00100000:1.0:1789762929.674225:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762929.674271:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015129472 00000100:00100000:0.0:1789762929.674277:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015129472 00000100:00100000:0.0:1789762929.674313:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453165731241 time 452971838645 state 0x0 ref 1 00000100:00100000:0.0:1789762929.674321:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453171838645 time 452971838645 state 0x1 ref 1 00000100:00100000:3.0:1789762929.868152:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453171838645 time 453172036662 state 0x0 ref 1 00000100:00100000:3.0:1789762929.868202:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015129472:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762929.868255:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05837 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.868273:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762929.868277:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762929.868283:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05837 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.868298:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05837 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.871923:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05837 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.871950:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05837 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.871959:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05837 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.871965:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762929.871970:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05837 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.871978:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05837 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.871999:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05837 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762929.872007:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05837 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762929.872071:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015129472:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3890us (197846us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762929.872080:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1260 00000100:00100000:3.0:1789762929.872136:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453171838645 time 453172036662 state 0x2 ref 0 00000100:00100000:2.0:1789762929.874477:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762929.874556:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015129600 00000100:00100000:1.0:1789762929.874566:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015129600 00000100:00100000:1.0:1789762929.874635:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453171838645 time 453172036662 state 0x0 ref 1 00000100:00100000:1.0:1789762929.874651:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453372036662 time 453172036662 state 0x1 ref 1 00000100:00100000:2.0:1789762930.068235:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453372036662 time 453372129609 state 0x0 ref 1 00000100:00100000:2.0:1789762930.068261:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015129600:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762930.068298:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006817d430/0x416193e012b0583e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762930.068312:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762930.068316:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762930.068320:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006817d430/0x416193e012b0583e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762930.068332:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006817d430/0x416193e012b0583e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.071654:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006817d430/0x416193e012b0583e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.071665:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006817d430/0x416193e012b0583e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.071672:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006817d430/0x416193e012b0583e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.071678:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762930.071681:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006817d430/0x416193e012b0583e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.071688:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006817d430/0x416193e012b0583e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.071710:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006817d430/0x416193e012b0583e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.071716:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006817d430/0x416193e012b0583e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762930.071764:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015129600:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3514us (197305us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762930.071771:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1261 00000100:00100000:0.0:1789762930.071779:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453372036662 time 453372129609 state 0x2 ref 0 00000100:00100000:2.0:1789762930.073921:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762930.073977:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015129728 00000100:00100000:0.0:1789762930.073985:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015129728 00000100:00100000:0.0:1789762930.074020:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453372036662 time 453372129609 state 0x0 ref 1 00000100:00100000:0.0:1789762930.074031:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453572129609 time 453372129609 state 0x1 ref 1 00000100:00100000:1.0:1789762930.268323:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453572129609 time 453572217377 state 0x0 ref 1 00000100:00100000:1.0:1789762930.268350:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84543480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015129728:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762930.268408:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000eb7be76a/0x416193e012b05845 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762930.268423:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762930.268426:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762930.268432:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000eb7be76a/0x416193e012b05845 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762930.268444:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000eb7be76a/0x416193e012b05845 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762930.272378:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000eb7be76a/0x416193e012b05845 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762930.272389:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000eb7be76a/0x416193e012b05845 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762930.272411:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000eb7be76a/0x416193e012b05845 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762930.272418:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7d200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762930.272422:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb7be76a/0x416193e012b05845 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762930.272428:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000eb7be76a/0x416193e012b05845 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762930.272445:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000eb7be76a/0x416193e012b05845 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762930.272452:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb7be76a/0x416193e012b05845 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762930.272500:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84543480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015129728:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4161us (198581us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762930.272508:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1262 00000100:00100000:3.0:1789762930.272516:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453572129609 time 453572217377 state 0x2 ref 0 00000100:00100000:2.0:1789762930.275790:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762930.275866:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015129856 00000100:00100000:0.0:1789762930.275875:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015129856 00000100:00100000:0.0:1789762930.275922:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453572129609 time 453572217377 state 0x0 ref 1 00000100:00100000:0.0:1789762930.275938:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453772217377 time 453572217377 state 0x1 ref 1 00000100:00100000:3.0:1789762930.468473:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453772217377 time 453772362292 state 0x0 ref 1 00000100:00100000:3.0:1789762930.468512:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84541880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015129856:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762930.468560:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0584c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762930.468577:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762930.468582:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762930.468587:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0584c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762930.468606:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0584c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.471330:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0584c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.471353:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0584c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.471360:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0584c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.471365:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762930.471369:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0584c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.471375:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0584c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.471405:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0584c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.471411:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0584c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762930.471466:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84541880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015129856:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2971us (195673us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762930.471474:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1263 00000100:00100000:0.0:1789762930.471488:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453772217377 time 453772362292 state 0x2 ref 0 00000100:00100000:2.0:1789762930.473277:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762930.473338:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015129984 00000100:00100000:1.0:1789762930.473346:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015129984 00000100:00100000:1.0:1789762930.473386:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453772217377 time 453772362292 state 0x0 ref 1 00000100:00100000:1.0:1789762930.473419:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453972362292 time 453772362292 state 0x1 ref 1 00010000:00010000:3.0:1789762930.615249:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762930.668574:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453972362292 time 453972470450 state 0x0 ref 1 00000100:00100000:0.0:1789762930.668614:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84542a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015129984:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762930.668652:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05853 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.668665:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762930.668669:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762930.668674:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05853 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.668686:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05853 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.672084:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05853 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.672119:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05853 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.672130:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05853 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.672139:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762930.672144:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05853 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.672156:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05853 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.672181:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05853 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762930.672187:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05853 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762930.672235:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84542a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015129984:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3630us (198961us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762930.672242:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1264 00000100:00100000:0.0:1789762930.672251:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453972362292 time 453972470450 state 0x2 ref 0 00000100:00100000:2.0:1789762930.674039:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762930.674076:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015130112 00000100:00100000:0.0:1789762930.674082:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015130112 00000100:00100000:0.0:1789762930.674139:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 453972362292 time 453972470450 state 0x0 ref 1 00000100:00100000:0.0:1789762930.674154:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454172470450 time 453972470450 state 0x1 ref 1 00000100:00100000:1.0:1789762930.786471:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762930.786506:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762930.786524:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015130240 00000100:00100000:1.0:1789762930.786531:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762930.786534:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015130240 00000100:00100000:1.0:1789762930.786556:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762930.786573:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015130368 00000100:00100000:0.0:1789762930.786579:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015130368 00000100:00100000:1.0:1789762930.786583:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015130624 00000100:00100000:1.0:1789762930.786590:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015130624 00000100:00100000:3.0:1789762930.786591:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 350 00000100:00100000:2.0:1789762930.786594:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015130496 00000100:00100000:2.0:1789762930.786602:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015130496 00000100:00100000:3.0:1789762930.786607:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015130240:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762930.786642:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 362 00000100:00100000:2.0:1789762930.786651:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 182 00000100:00100000:0.0:1789762930.786657:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2366:x1876702015130368:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762930.786663:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 212 00000100:00100000:2.0:1789762930.786668:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2365:x1876702015130496:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762930.786675:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015130240:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 71us (208us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762930.786679:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2362:x1876702015130624:12345-192.168.203.48@tcp:9:kworker.0 00000100:00100000:3.0:1789762930.786686:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 350 00000020:00000080:1.0:1789762930.786702:0:6648:0:(genops.c:1468:class_disconnect()) disconnect: cookie 0x416193e012b05305 00000100:00100000:0.0:1789762930.786708:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2366:x1876702015130368:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 56us (200us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762930.786713:0:6648:0:(import.c:65:import_set_state_nolock()) ffff981d8a646000 : changing import state from FULL to CLOSED 00000100:00100000:2.0:1789762930.786716:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2365:x1876702015130496:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 52us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762930.786719:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 362 00000100:00080000:3.0:1789762930.786720:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762930.786728:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 182 00000001:00000004:1.0:1789762930.786751:0:6648:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967434 00010000:00010000:0.0:1789762930.786758:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762930.786762:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762930.786792:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015130752 00000100:00100000:2.0:1789762930.786796:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015130752 00000100:00080000:0.0:1789762930.786845:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762930.786856:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 363 00000100:00100000:2.0:1789762930.786868:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015130752:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762930.786908:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015130752:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 45us (151us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762930.786914:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 363 00000100:00100000:1.0:1789762930.787038:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+2:2362:x1876702015130624:12345-192.168.203.48@tcp:9:kworker.0 Request processed in 364us (506us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762930.787044:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 212 00000100:00100000:0.0:1789762930.807254:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022193664:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762930.807299:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762930.807318:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022193664 00000100:00100000:2.0:1789762930.807324:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022193664 00000100:00080000:3.0:1789762930.807378:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789762930.807383:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:2.0:1789762930.807407:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022193664:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789762930.807431:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:2.0:1789762930.807469:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb25180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022193664:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (171us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762930.807475:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00100000:0.0:1789762930.807518:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022193664:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762930.868655:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454172470450 time 454172550225 state 0x0 ref 1 00000100:00100000:3.0:1789762930.868677:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84542680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015130112:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762930.868712:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000071770fd6/0x416193e012b0585a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762930.868726:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762930.868729:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762930.868750:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000071770fd6/0x416193e012b0585a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762930.868765:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000071770fd6/0x416193e012b0585a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762930.871768:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000071770fd6/0x416193e012b0585a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762930.871780:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000071770fd6/0x416193e012b0585a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762930.871786:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000071770fd6/0x416193e012b0585a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762930.871791:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7c400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762930.871795:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000071770fd6/0x416193e012b0585a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762930.871802:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000071770fd6/0x416193e012b0585a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762930.871825:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000071770fd6/0x416193e012b0585a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762930.871835:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071770fd6/0x416193e012b0585a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762930.871896:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84542680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015130112:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3220us (197852us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762930.871904:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1265 00000100:00100000:1.0:1789762930.871912:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454172470450 time 454172550225 state 0x2 ref 0 00000100:00100000:0.0:1789762930.874029:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762930.874082:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015130880 00000100:00100000:1.0:1789762930.874106:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015130880 00000100:00100000:1.0:1789762930.874142:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454172470450 time 454172550225 state 0x0 ref 1 00000100:00100000:1.0:1789762930.874153:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454372550225 time 454172550225 state 0x1 ref 1 00000100:00100000:0.0:1789762931.068764:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454372550225 time 454372655852 state 0x0 ref 1 00000100:00100000:0.0:1789762931.068819:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015130880:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:3.0:1789762931.068856:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762931.068884:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05861 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.068901:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762931.068905:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762931.068910:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05861 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.068921:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05861 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.073593:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05861 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.073611:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05861 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.073643:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05861 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.073654:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7a400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762931.073660:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05861 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.073672:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05861 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.073692:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05861 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.073701:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05861 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762931.073766:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb24000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015130880:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4983us (199735us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762931.073778:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1266 00000100:00100000:0.0:1789762931.073790:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454372550225 time 454372655852 state 0x2 ref 0 00000100:00100000:0.0:1789762931.076350:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762931.076456:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015131008 00000100:00100000:1.0:1789762931.076464:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015131008 00000100:00100000:1.0:1789762931.076502:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454372550225 time 454372655852 state 0x0 ref 1 00000100:00100000:1.0:1789762931.076538:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454572655852 time 454372655852 state 0x1 ref 1 00000100:00100000:0.0:1789762931.268905:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454572655852 time 454572794740 state 0x0 ref 1 00000100:00100000:0.0:1789762931.268940:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb25500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015131008:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762931.269004:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05868 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.269022:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762931.269027:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762931.269033:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05868 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.269048:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05868 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.273138:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05868 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.273152:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05868 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.273159:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05868 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.273165:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7a400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762931.273176:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05868 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.273184:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05868 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.273213:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05868 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.273220:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05868 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762931.273283:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb25500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015131008:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4358us (196936us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762931.273291:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1267 00000100:00100000:0.0:1789762931.273300:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454572655852 time 454572794740 state 0x2 ref 0 00000100:00100000:0.0:1789762931.275688:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762931.275734:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015131136 00000100:00100000:3.0:1789762931.275743:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015131136 00000100:00100000:3.0:1789762931.275785:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454572655852 time 454572794740 state 0x0 ref 1 00000100:00100000:3.0:1789762931.275798:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454772794740 time 454572794740 state 0x1 ref 1 00000100:00100000:0.0:1789762931.319238:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789762931.319239:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789762931.319255:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb25c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022193920:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762931.319262:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d84540380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022193792:0@lo:400:kworker.0 00000100:00100000:3.0:1789762931.319264:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac5b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022194304:0@lo:400:kworker.0 00000100:00100000:2.0:1789762931.319288:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac58380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022194048:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.319355:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762931.319363:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762931.319373:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762931.319402:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022193792 00000100:00100000:3.0:1789762931.319413:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022193792 00000100:00100000:1.0:1789762931.319443:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac59880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022194432:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.319462:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022193920 00000100:00100000:0.0:1789762931.319471:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022193920 00000100:00100000:2.0:1789762931.319471:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762931.319477:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 351 00000100:00100000:3.0:1789762931.319491:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022193792:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762931.319498:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762931.319527:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebac58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022194176:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.319534:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:0.0:1789762931.319547:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022193920:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762931.319565:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022194432 00000100:00100000:3.0:1789762931.319571:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3273:x1876702022193792:12345-0@lo:400:kworker.0 Request processed in 82us (212us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762931.319573:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022194432 00002000:00000024:0.0:1789762931.319579:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762931.319579:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762931.319588:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 351 00000100:00100000:1.0:1789762931.319623:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:1.0:1789762931.319634:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022194432:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.319652:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022193920:12345-0@lo:13:osp-pre-1-0.0 Request processed in 106us (290us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762931.319659:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022194048 00000100:00100000:3.0:1789762931.319660:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022194304 00000100:00100000:0.0:1789762931.319667:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1789762931.319669:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022194048 00000100:00100000:3.0:1789762931.319670:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022194304 00000100:00100000:1.0:1789762931.319678:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022194432:12345-0@lo:400:kworker.0 Request processed in 45us (179us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762931.319687:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:2.0:1789762931.319723:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022194176 00000100:00100000:2.0:1789762931.319728:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022194176 00000100:00100000:0.0:1789762931.319734:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac59880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022194432:0@lo:400:kworker.0 00000100:00100000:3.0:1789762931.319746:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:1.0:1789762931.319758:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:3.0:1789762931.319764:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022194048:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762931.319775:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022194304:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.319779:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb25c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022193920:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762931.319787:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:2.0:1789762931.319801:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022194176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762931.319827:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022194048:12345-0@lo:400:kworker.0 Request processed in 69us (456us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762931.319836:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:1.0:1789762931.319845:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022194304:12345-0@lo:400:kworker.0 Request processed in 76us (380us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762931.319854:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00080000:3.0:1789762931.319867:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762931.319869:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022194176:12345-0@lo:400:kworker.0 Request processed in 63us (286us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762931.319873:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762931.319880:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000100:00100000:0.0:1789762931.319901:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac5b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022194304:0@lo:400:kworker.0 00000100:00100000:3.0:1789762931.319917:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d84540380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3273:1876702022193792:0@lo:400:kworker.0 00000100:00100000:0.0:1789762931.319928:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac58700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022194176:0@lo:400:kworker.0 00000100:00100000:2.0:1789762931.319935:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebac58380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022194048:0@lo:400:kworker.0 00000100:00100000:1.0:1789762931.469020:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454772794740 time 454772911581 state 0x0 ref 1 00000100:00100000:1.0:1789762931.469048:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015131136:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762931.469144:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0586f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762931.469162:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762931.469167:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762931.469173:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0586f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762931.469195:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0586f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762931.475247:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0586f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762931.475269:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0586f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762931.475286:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0586f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762931.475300:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762931.475309:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0586f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762931.475324:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0586f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762931.475350:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0586f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762931.475363:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0586f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762931.475516:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb25f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015131136:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6471us (199821us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762931.475536:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1268 00000100:00100000:1.0:1789762931.475557:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454772794740 time 454772911581 state 0x2 ref 0 00000100:00100000:2.0:1789762931.478522:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762931.478594:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015131264 00000100:00100000:1.0:1789762931.478608:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015131264 00000100:00100000:1.0:1789762931.478682:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454772794740 time 454772911581 state 0x0 ref 1 00000100:00100000:1.0:1789762931.478701:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454972911581 time 454772911581 state 0x1 ref 1 00010000:00010000:0.0:1789762931.639195:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762931.669147:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454972911581 time 454973037835 state 0x0 ref 1 00000100:00100000:0.0:1789762931.669179:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015131264:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762931.669224:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05876 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.669242:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762931.669247:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762931.669254:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05876 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.669269:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05876 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.673288:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05876 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.673303:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05876 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.673313:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05876 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.673332:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7a400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762931.673338:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05876 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.673347:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05876 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.673364:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05876 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.673372:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b05876 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762931.673456:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015131264:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4288us (194940us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762931.673466:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1269 00000100:00100000:0.0:1789762931.673476:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454972911581 time 454973037835 state 0x2 ref 0 00000100:00100000:3.0:1789762931.676432:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762931.676498:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015131392 00000100:00100000:1.0:1789762931.676510:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015131392 00000100:00100000:1.0:1789762931.676559:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 454972911581 time 454973037835 state 0x0 ref 1 00000100:00100000:1.0:1789762931.676574:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455173037835 time 454973037835 state 0x1 ref 1 00000100:00100000:0.0:1789762931.869271:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455173037835 time 455173162603 state 0x0 ref 1 00000100:00100000:0.0:1789762931.869301:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015131392:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762931.869342:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0587d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.869377:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762931.869396:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762931.869404:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0587d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.869420:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0587d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.873151:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0587d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.873168:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0587d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.873181:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0587d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.873191:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7a400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762931.873198:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0587d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.873210:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0587d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.873232:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0587d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762931.873259:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0587d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762931.873329:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015131392:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4039us (196898us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762931.873342:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1270 00000100:00100000:0.0:1789762931.873355:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455173037835 time 455173162603 state 0x2 ref 0 00000100:00100000:3.0:1789762931.875791:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762931.875839:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015131520 00000100:00100000:0.0:1789762931.875849:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015131520 00000100:00100000:0.0:1789762931.875893:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455173037835 time 455173162603 state 0x0 ref 1 00000100:00100000:0.0:1789762931.875910:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455373162603 time 455173162603 state 0x1 ref 1 00000100:00100000:1.0:1789762932.069510:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455373162603 time 455373391164 state 0x0 ref 1 00000100:00100000:1.0:1789762932.069555:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84540380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015131520:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762932.069613:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05884 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.069640:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762932.069649:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762932.069660:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05884 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.069684:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05884 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.074323:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05884 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.074344:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05884 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.074358:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05884 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.074370:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762932.074376:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05884 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.074409:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05884 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.074433:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05884 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.074446:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05884 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762932.074527:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84540380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015131520:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4990us (198736us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762932.074543:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1271 00000100:00100000:1.0:1789762932.074557:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455373162603 time 455373391164 state 0x2 ref 0 00000100:00100000:2.0:1789762932.077139:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762932.077180:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015131648 00000100:00100000:1.0:1789762932.077192:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015131648 00000100:00100000:1.0:1789762932.077244:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455373162603 time 455373391164 state 0x0 ref 1 00000100:00100000:1.0:1789762932.077263:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455573391164 time 455373391164 state 0x1 ref 1 00000100:00100000:0.0:1789762932.269661:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455573391164 time 455573544786 state 0x0 ref 1 00000100:00100000:0.0:1789762932.269693:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015131648:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762932.269736:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0588b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762932.269754:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762932.269758:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762932.269764:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0588b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762932.269798:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0588b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.273638:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0588b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.273659:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0588b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.273673:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0588b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.273686:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7a400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762932.273693:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0588b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.273707:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0588b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.273727:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0588b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.273734:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003c23d9c2/0x416193e012b0588b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762932.273796:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015131648:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4115us (196659us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762932.273810:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1272 00000100:00100000:3.0:1789762932.273824:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455573391164 time 455573544786 state 0x2 ref 0 00000100:00100000:2.0:1789762932.276199:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762932.276259:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015131776 00000100:00100000:1.0:1789762932.276270:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015131776 00000100:00100000:1.0:1789762932.276315:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455573391164 time 455573544786 state 0x0 ref 1 00000100:00100000:1.0:1789762932.276328:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455773544786 time 455573544786 state 0x1 ref 1 00000100:00100000:0.0:1789762932.469741:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455773544786 time 455773632119 state 0x0 ref 1 00000100:00100000:0.0:1789762932.469770:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015131776:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762932.469813:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b05892 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762932.469833:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762932.469838:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762932.469844:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b05892 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762932.469858:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b05892 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762932.473569:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b05892 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762932.473579:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b05892 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762932.473586:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b05892 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762932.473600:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7be00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762932.473604:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b05892 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762932.473611:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b05892 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762932.473626:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b05892 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762932.473631:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b05892 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762932.473675:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015131776:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3915us (197478us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762932.473683:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1273 00000100:00100000:0.0:1789762932.473690:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455773544786 time 455773632119 state 0x2 ref 0 00000100:00100000:0.0:1789762932.475788:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762932.475843:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015131904 00000100:00100000:3.0:1789762932.475853:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015131904 00000100:00100000:3.0:1789762932.475898:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455773544786 time 455773632119 state 0x0 ref 1 00000100:00100000:3.0:1789762932.475913:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455973632119 time 455773632119 state 0x1 ref 1 00010000:00010000:3.0:1789762932.663268:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762932.669884:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455973632119 time 455973776669 state 0x0 ref 1 00000100:00100000:1.0:1789762932.669919:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015131904:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762932.669972:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05899 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.669998:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762932.670007:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762932.670018:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05899 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.670041:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05899 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.674300:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05899 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.674316:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05899 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.674326:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05899 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.674335:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762932.674341:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05899 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.674352:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05899 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.674393:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05899 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762932.674404:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b05899 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762932.674510:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb25c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015131904:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4597us (198715us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762932.674527:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1274 00000100:00100000:1.0:1789762932.674545:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455973632119 time 455973776669 state 0x2 ref 0 00000100:00100000:0.0:1789762932.677574:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762932.677654:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015132032 00000100:00100000:1.0:1789762932.677665:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015132032 00000100:00100000:1.0:1789762932.677733:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 455973632119 time 455973776669 state 0x0 ref 1 00000100:00100000:1.0:1789762932.677758:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456173776669 time 455973776669 state 0x1 ref 1 00000100:00100000:3.0:1789762932.870899:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456173776669 time 456174790120 state 0x0 ref 1 00000100:00100000:3.0:1789762932.870946:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015132032:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762932.870998:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000056ca7d58/0x416193e012b058a0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.871017:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762932.871024:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762932.871032:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056ca7d58/0x416193e012b058a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.871053:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056ca7d58/0x416193e012b058a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00100000:2.0:1789762932.871171:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:2.0:1789762932.871181:0:7605:0:(genops.c:885:class_export_put()) final put ffff981d8a645800/f1bd28de-ee78-4d18-bad3-fa7cec968893 00000020:00000080:0.0:1789762932.871241:0:36:0:(genops.c:844:class_export_destroy()) destroying export ffff981d8a645800/f1bd28de-ee78-4d18-bad3-fa7cec968893 for lustre-OST0001 00000020:00000080:0.0:1789762932.871324:0:36:0:(genops.c:1183:obd_zombie_import_free()) destroying import ffff981d8a646000 for lustre-OST0001 00010000:00010000:3.0:1789762932.875576:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000056ca7d58/0x416193e012b058a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.875594:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000056ca7d58/0x416193e012b058a0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.875605:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000056ca7d58/0x416193e012b058a0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.875614:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a79e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762932.875619:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000056ca7d58/0x416193e012b058a0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.875629:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000056ca7d58/0x416193e012b058a0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.875647:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000056ca7d58/0x416193e012b058a0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762932.875656:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000056ca7d58/0x416193e012b058a0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762932.875733:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb24700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015132032:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4810us (198160us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762932.875751:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1275 00000100:00100000:3.0:1789762932.875768:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456173776669 time 456174790120 state 0x2 ref 0 00000100:00100000:3.0:1789762932.878454:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762932.878530:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015132160 00000100:00100000:1.0:1789762932.878542:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015132160 00000100:00100000:1.0:1789762932.878595:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456173776669 time 456174790120 state 0x0 ref 1 00000100:00100000:1.0:1789762932.878612:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456374790120 time 456174790120 state 0x1 ref 1 00000100:00100000:0.0:1789762933.071066:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456374790120 time 456374944001 state 0x0 ref 1 00000100:00100000:0.0:1789762933.071200:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015132160:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:3.0:1789762933.071214:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762933.071270:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058a7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.071295:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762933.071301:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762933.071310:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058a7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.071329:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058a7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.077564:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058a7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.077584:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058a7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.077597:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058a7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.077608:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7be00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762933.077615:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058a7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.077646:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058a7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.077670:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058a7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.077680:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058a7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762933.077758:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015132160:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6595us (199305us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762933.077771:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1276 00000100:00100000:0.0:1789762933.077786:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456374790120 time 456374944001 state 0x2 ref 0 00000100:00100000:3.0:1789762933.081472:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762933.081540:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015132288 00000100:00100000:0.0:1789762933.081554:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015132288 00000100:00100000:0.0:1789762933.081615:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456374790120 time 456374944001 state 0x0 ref 1 00000100:00100000:0.0:1789762933.081635:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456574944001 time 456374944001 state 0x1 ref 1 00000100:00100000:1.0:1789762933.271293:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456574944001 time 456575178914 state 0x0 ref 1 00000100:00100000:1.0:1789762933.271334:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015132288:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762933.271413:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058ae lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.271437:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762933.271444:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762933.271453:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058ae lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.271471:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058ae lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.276855:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058ae lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.276875:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058ae lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.276888:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058ae lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.276898:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762933.276904:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058ae lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.276915:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058ae lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.276937:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058ae lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.276947:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058ae lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762933.277036:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015132288:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5714us (195568us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762933.277056:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1277 00000100:00100000:1.0:1789762933.277073:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456574944001 time 456575178914 state 0x2 ref 0 00000100:00100000:3.0:1789762933.280478:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762933.280541:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015132416 00000100:00100000:1.0:1789762933.280556:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015132416 00000100:00100000:1.0:1789762933.280613:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456574944001 time 456575178914 state 0x0 ref 1 00000100:00100000:1.0:1789762933.280633:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456775178914 time 456575178914 state 0x1 ref 1 00000100:00100000:3.0:1789762933.471507:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456775178914 time 456775392781 state 0x0 ref 1 00000100:00100000:3.0:1789762933.471542:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015132416:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762933.471607:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b058b5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.471640:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762933.471648:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762933.471661:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b058b5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762933.471689:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b058b5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.477442:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b058b5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.477471:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b058b5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.477510:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b058b5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.477522:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a7200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762933.477529:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b058b5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.477541:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b058b5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.477564:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b058b5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.477574:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000066fa5b38/0x416193e012b058b5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762933.477663:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015132416:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6129us (197187us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762933.477689:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1278 00000100:00100000:0.0:1789762933.477704:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456775178914 time 456775392781 state 0x2 ref 0 00000100:00100000:2.0:1789762933.482616:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762933.482715:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015132544 00000100:00100000:1.0:1789762933.482729:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015132544 00000100:00100000:1.0:1789762933.482803:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456775178914 time 456775392781 state 0x0 ref 1 00000100:00100000:1.0:1789762933.482842:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456975392781 time 456775392781 state 0x1 ref 1 00000100:00100000:0.0:1789762933.671691:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456975392781 time 456975576178 state 0x0 ref 1 00000100:00100000:0.0:1789762933.671732:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015132544:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762933.671812:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058bc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.671842:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762933.671851:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762933.671863:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058bc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762933.671907:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058bc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.677591:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058bc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.677621:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058bc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.677642:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058bc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.677685:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7be00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762933.677694:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058bc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.677714:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058bc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.677746:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058bc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.677763:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000096e5d68d/0x416193e012b058bc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.677884:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015132544:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6164us (195271us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762933.677906:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1279 00000100:00100000:2.0:1789762933.677927:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456975392781 time 456975576178 state 0x2 ref 0 00000100:00100000:2.0:1789762933.681502:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762933.681589:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015132672 00000100:00100000:0.0:1789762933.681603:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015132672 00000100:00100000:0.0:1789762933.681662:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 456975392781 time 456975576178 state 0x0 ref 1 00000100:00100000:0.0:1789762933.681681:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457175576178 time 456975576178 state 0x1 ref 1 00010000:00010000:3.0:1789762933.687403:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762933.871891:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457175576178 time 457175772768 state 0x0 ref 1 00000100:00100000:1.0:1789762933.871956:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015132672:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762933.872025:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058c3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.872048:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762933.872056:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762933.872063:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058c3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762933.872154:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058c3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.882555:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058c3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.882583:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058c3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.882619:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058c3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.882631:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762933.882639:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058c3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.882653:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058c3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.882683:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058c3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762933.882699:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b058c3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762933.882817:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015132672:12345-192.168.203.48@tcp:3:dd.500 Request processed in 10891us (201313us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762933.882838:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1280 00000100:00100000:2.0:1789762933.882855:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457175576178 time 457175772768 state 0x2 ref 0 00000100:00100000:3.0:1789762933.886811:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762933.886873:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015132800 00000100:00100000:1.0:1789762933.886891:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015132800 00000100:00100000:1.0:1789762933.886976:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457175576178 time 457175772768 state 0x0 ref 1 00000100:00100000:1.0:1789762933.887001:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457375772768 time 457175772768 state 0x1 ref 1 00000001:00080000:0.0:1789762933.895414:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967607 is committed 00000001:00080000:0.0:1789762933.895435:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967608 is committed 00000100:00100000:0.0:1789762934.072040:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457375772768 time 457375920841 state 0x0 ref 1 00000100:00100000:0.0:1789762934.072129:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015132800:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762934.072196:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058ca lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.072226:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762934.072259:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762934.072268:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058ca lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.072288:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058ca lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.077585:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058ca lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.077609:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058ca lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.077629:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058ca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.077647:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762934.077658:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058ca lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.077677:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058ca lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.077704:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058ca lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.077716:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058ca lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762934.077833:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015132800:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5757us (191022us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762934.077852:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1281 00000100:00100000:0.0:1789762934.077873:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457375772768 time 457375920841 state 0x2 ref 0 00000100:00100000:3.0:1789762934.081082:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762934.081205:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015132928 00000100:00100000:1.0:1789762934.081222:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015132928 00000100:00100000:1.0:1789762934.081285:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457375772768 time 457375920841 state 0x0 ref 1 00000100:00100000:1.0:1789762934.081304:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457575920841 time 457375920841 state 0x1 ref 1 00000100:00100000:2.0:1789762934.272217:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457575920841 time 457576102248 state 0x0 ref 1 00000100:00100000:2.0:1789762934.272257:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015132928:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762934.272337:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.272395:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.272406:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.272419:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.272449:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.278290:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.278324:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.278344:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.278387:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe19200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762934.278398:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.278416:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.278465:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.278477:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762934.278567:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015132928:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6321us (197484us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762934.278586:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1282 00000100:00100000:2.0:1789762934.278604:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457575920841 time 457576102248 state 0x2 ref 0 00000100:00100000:2.0:1789762934.282010:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762934.282151:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015133056 00000100:00100000:0.0:1789762934.282168:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015133056 00000100:00100000:0.0:1789762934.282229:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457575920841 time 457576102248 state 0x0 ref 1 00000100:00100000:0.0:1789762934.282248:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457776102248 time 457576102248 state 0x1 ref 1 00000100:00100000:2.0:1789762934.472453:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457776102248 time 457776339665 state 0x0 ref 1 00000100:00100000:2.0:1789762934.472486:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015133056:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762934.472539:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.472562:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.472568:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.472578:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.472596:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.477590:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.477620:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.477642:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.477671:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe19200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762934.477678:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.477691:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.477712:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762934.477722:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a805a2f6/0x416193e012b058d8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762934.477813:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015133056:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5336us (195805us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762934.477827:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1283 00000100:00100000:1.0:1789762934.477841:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457776102248 time 457776339665 state 0x2 ref 0 00000100:00100000:3.0:1789762934.481563:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762934.481627:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015133184 00000100:00100000:1.0:1789762934.481640:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015133184 00000100:00100000:1.0:1789762934.481696:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457776102248 time 457776339665 state 0x0 ref 1 00000100:00100000:1.0:1789762934.481715:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457976339665 time 457776339665 state 0x1 ref 1 00000100:00100000:0.0:1789762934.672635:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457976339665 time 457976520301 state 0x0 ref 1 00000100:00100000:0.0:1789762934.672676:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015133184:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762934.672746:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058df lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.672770:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762934.672776:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762934.672785:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058df lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.672803:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058df lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.678233:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058df lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.678254:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058df lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.678267:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058df lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.678278:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762934.678286:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058df lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.678298:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058df lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.678322:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058df lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.678335:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058df lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762934.678455:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015133184:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5794us (196893us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762934.678468:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1284 00000100:00100000:0.0:1789762934.678494:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457976339665 time 457976520301 state 0x2 ref 0 00000100:00100000:3.0:1789762934.682224:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762934.682314:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015133312 00000100:00100000:1.0:1789762934.682329:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015133312 00000100:00100000:1.0:1789762934.682425:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 457976339665 time 457976520301 state 0x0 ref 1 00000100:00100000:1.0:1789762934.682454:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458176520301 time 457976520301 state 0x1 ref 1 00010000:00010000:0.0:1789762934.711249:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762934.872852:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458176520301 time 458176730411 state 0x0 ref 1 00000100:00100000:2.0:1789762934.872898:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015133312:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762934.872956:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000055b06452/0x416193e012b058e6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.872993:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762934.872999:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762934.873008:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000055b06452/0x416193e012b058e6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762934.873027:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000055b06452/0x416193e012b058e6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.879000:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000055b06452/0x416193e012b058e6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.879029:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000055b06452/0x416193e012b058e6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.879049:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000055b06452/0x416193e012b058e6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.879064:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e84a7aa00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762934.879073:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000055b06452/0x416193e012b058e6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.879195:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000055b06452/0x416193e012b058e6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.879232:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000055b06452/0x416193e012b058e6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762934.879250:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000055b06452/0x416193e012b058e6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762934.879418:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015133312:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6529us (197189us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762934.879460:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1285 00000100:00100000:0.0:1789762934.879475:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458176520301 time 458176730411 state 0x2 ref 0 00000100:00100000:3.0:1789762934.881782:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762934.881850:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015133440 00000100:00100000:0.0:1789762934.881863:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015133440 00000100:00100000:0.0:1789762934.881938:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458176520301 time 458176730411 state 0x0 ref 1 00000100:00100000:0.0:1789762934.881967:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458376730411 time 458176730411 state 0x1 ref 1 00000100:00100000:1.0:1789762935.073019:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458376730411 time 458376905908 state 0x0 ref 1 00000100:00100000:1.0:1789762935.073084:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015133440:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:3.0:1789762935.073145:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762935.073174:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b058ed lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.073200:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762935.073206:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762935.073215:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b058ed lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.073234:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b058ed lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.077810:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b058ed lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.077830:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b058ed lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.077843:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b058ed lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.077856:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762935.077863:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b058ed lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.077875:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b058ed lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.077899:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b058ed lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.077909:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b058ed lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762935.077988:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015133440:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4944us (196209us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762935.078002:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1286 00000100:00100000:1.0:1789762935.078016:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458376730411 time 458376905908 state 0x2 ref 0 00000100:00100000:3.0:1789762935.080991:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762935.081129:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015133568 00000100:00100000:1.0:1789762935.081143:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015133568 00000100:00100000:1.0:1789762935.081202:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458376730411 time 458376905908 state 0x0 ref 1 00000100:00100000:1.0:1789762935.081221:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458576905908 time 458376905908 state 0x1 ref 1 00000100:00100000:0.0:1789762935.273185:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458576905908 time 458577070645 state 0x0 ref 1 00000100:00100000:0.0:1789762935.273224:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015133568:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762935.273280:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058f4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.273303:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762935.273309:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762935.273318:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058f4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.273336:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058f4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.277995:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058f4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.278056:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058f4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.278078:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058f4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.278130:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762935.278137:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058f4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.278151:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058f4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.278173:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058f4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.278184:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058f4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762935.278270:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015133568:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5059us (197279us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762935.278284:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1287 00000100:00100000:0.0:1789762935.278297:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458576905908 time 458577070645 state 0x2 ref 0 00000100:00100000:2.0:1789762935.282074:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762935.282184:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015133696 00000100:00100000:1.0:1789762935.282199:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015133696 00000100:00100000:1.0:1789762935.282258:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458576905908 time 458577070645 state 0x0 ref 1 00000100:00100000:1.0:1789762935.282278:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458777070645 time 458577070645 state 0x1 ref 1 00000100:00100000:0.0:1789762935.473329:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458777070645 time 458777207668 state 0x0 ref 1 00000100:00100000:0.0:1789762935.473445:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015133696:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762935.473512:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058fb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.473543:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762935.473551:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762935.473561:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058fb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.473580:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058fb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.478852:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058fb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.478881:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058fb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.478903:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058fb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.478933:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762935.478940:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058fb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.478974:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058fb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.478997:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058fb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.479019:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b058fb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762935.479157:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac58700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015133696:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5745us (197080us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762935.479177:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1288 00000100:00100000:0.0:1789762935.479199:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458777070645 time 458777207668 state 0x2 ref 0 00000100:00100000:3.0:1789762935.482659:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762935.482747:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015133824 00000100:00100000:0.0:1789762935.482762:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015133824 00000100:00100000:0.0:1789762935.482823:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458777070645 time 458777207668 state 0x0 ref 1 00000100:00100000:0.0:1789762935.482850:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458977207668 time 458777207668 state 0x1 ref 1 00000100:00100000:1.0:1789762935.673540:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458977207668 time 458977419665 state 0x0 ref 1 00000100:00100000:1.0:1789762935.673590:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015133824:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762935.673658:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05902 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.673690:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762935.673699:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762935.673711:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05902 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.673737:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05902 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.679697:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05902 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.679717:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05902 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.679729:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05902 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.679740:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762935.679746:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05902 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.679758:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05902 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.679781:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05902 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762935.679794:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05902 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762935.679928:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015133824:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6356us (197271us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762935.679941:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1289 00000100:00100000:1.0:1789762935.679955:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458977207668 time 458977419665 state 0x2 ref 0 00000100:00100000:3.0:1789762935.683288:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762935.683348:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015133952 00000100:00100000:1.0:1789762935.683393:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015133952 00000100:00100000:1.0:1789762935.683455:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 458977207668 time 458977419665 state 0x0 ref 1 00000100:00100000:1.0:1789762935.683474:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459177419665 time 458977419665 state 0x1 ref 1 00010000:00010000:0.0:1789762935.735242:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762935.873708:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459177419665 time 459177587888 state 0x0 ref 1 00000100:00100000:0.0:1789762935.873760:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015133952:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762935.873854:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05909 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.873881:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762935.873889:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762935.873901:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05909 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.873928:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05909 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.880790:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05909 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.880819:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05909 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.880839:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05909 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.880856:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762935.880866:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05909 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.880887:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05909 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.880944:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05909 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762935.880956:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05909 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762935.881048:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015133952:12345-192.168.203.48@tcp:3:dd.500 Request processed in 7306us (197759us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762935.881061:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1290 00000100:00100000:0.0:1789762935.881076:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459177419665 time 459177587888 state 0x2 ref 0 00000100:00100000:3.0:1789762935.885497:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762935.885586:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015134080 00000100:00100000:1.0:1789762935.885600:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015134080 00000100:00100000:1.0:1789762935.885663:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459177419665 time 459177587888 state 0x0 ref 1 00000100:00100000:1.0:1789762935.885684:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459377587888 time 459177587888 state 0x1 ref 1 00000100:00100000:2.0:1789762935.906755:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762935.906821:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762935.906851:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015134336 00000100:00100000:3.0:1789762935.906866:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015134336 00000100:00100000:0.0:1789762935.906933:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015134208 00000100:00100000:0.0:1789762935.906948:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015134208 00000100:00100000:2.0:1789762935.906974:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00080000:1.0:1789762935.906992:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762935.907005:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015134464 00000100:00100000:3.0:1789762935.907015:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015134464 00000100:00100000:0.0:1789762935.907036:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 352 00000100:00100000:1.0:1789762935.907046:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 367 00000100:00100000:0.0:1789762935.907062:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015134208:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762935.907077:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702015134336:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762935.907156:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 368 00000100:00100000:2.0:1789762935.907176:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015134464:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762935.907208:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2363:x1876702015134208:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 152us (384us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762935.907222:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 352 00000100:00100000:1.0:1789762935.907233:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac5b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2363:x1876702015134336:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 162us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762935.907251:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac59880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015134464:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 79us (281us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762935.907259:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 367 00000100:00100000:2.0:1789762935.907278:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 368 00000100:00080000:0.0:1789762935.907294:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762935.927339:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb19180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022194560:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762935.927446:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762935.927549:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022194560 00000100:00100000:0.0:1789762935.927562:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022194560 00000100:00080000:1.0:1789762935.927629:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762935.927649:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:0.0:1789762935.927665:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022194560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762935.927705:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762935.927809:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022194560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 146us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762935.927822:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1789762935.927869:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb19180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022194560:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762936.073897:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459377587888 time 459377779998 state 0x0 ref 1 00000100:00100000:0.0:1789762936.073938:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015134080:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762936.073992:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05910 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.074015:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762936.074023:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762936.074032:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05910 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.074051:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05910 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.078688:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05910 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.078705:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05910 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.078716:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05910 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.078725:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762936.078748:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05910 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.078760:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05910 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.078782:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05910 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.078793:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05910 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762936.078874:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015134080:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4951us (193380us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762936.078885:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1291 00000100:00100000:0.0:1789762936.078898:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459377587888 time 459377779998 state 0x2 ref 0 00000100:00100000:3.0:1789762936.083195:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762936.083262:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015134592 00000100:00100000:0.0:1789762936.083275:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015134592 00000100:00100000:0.0:1789762936.083334:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459377587888 time 459377779998 state 0x0 ref 1 00000100:00100000:0.0:1789762936.083389:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459577779998 time 459377779998 state 0x1 ref 1 00000100:00100000:1.0:1789762936.274053:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459577779998 time 459577938085 state 0x0 ref 1 00000100:00100000:1.0:1789762936.274123:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015134592:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762936.274181:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05917 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.274204:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762936.274211:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762936.274219:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05917 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.274238:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05917 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.279984:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05917 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.280001:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05917 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.280013:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05917 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.280023:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762936.280030:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05917 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.280042:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05917 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.280068:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05917 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.280078:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05917 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762936.280214:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015134592:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6133us (197027us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762936.280230:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1292 00000100:00100000:1.0:1789762936.280245:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459577779998 time 459577938085 state 0x2 ref 0 00000100:00100000:3.0:1789762936.284448:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762936.284509:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015134720 00000100:00100000:1.0:1789762936.284522:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015134720 00000100:00100000:1.0:1789762936.284577:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459577779998 time 459577938085 state 0x0 ref 1 00000100:00100000:1.0:1789762936.284595:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459777938085 time 459577938085 state 0x1 ref 1 00000100:00100000:0.0:1789762936.439344:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8aa7bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022194688:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762936.439442:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e902de680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022195072:0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.439472:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022194816:0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.439489:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762936.439511:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8387e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022194944:0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.439528:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762936.439603:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e902dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022195200:0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.439611:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762936.439627:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022195072 00000100:00100000:3.0:1789762936.439639:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762936.439642:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022195072 00000100:00100000:1.0:1789762936.439714:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e902dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022195328:0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.439725:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762936.439750:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022194944 00000100:00100000:2.0:1789762936.439758:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022194944 00000100:00100000:3.0:1789762936.439772:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022195200 00000100:00100000:3.0:1789762936.439787:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022195200 00000100:00100000:1.0:1789762936.439819:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762936.439832:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:0.0:1789762936.439834:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022194688 00000100:00100000:0.0:1789762936.439846:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022194688 00000100:00100000:2.0:1789762936.439857:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022195072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762936.439890:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:1.0:1789762936.439915:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022194816 00000100:00100000:3.0:1789762936.439919:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022194944:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.439925:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022194816 00000100:00100000:0.0:1789762936.439931:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1789762936.439957:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022194688:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762936.440010:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022195072:12345-0@lo:400:kworker.0 Request processed in 159us (486us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762936.440013:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 353 00002000:00000024:0.0:1789762936.440015:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762936.440043:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022194816:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.440044:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00000100:00100000:3.0:1789762936.440061:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902ded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022194944:12345-0@lo:400:kworker.0 Request processed in 146us (429us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762936.440143:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:2.0:1789762936.440160:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 371 00000100:00100000:0.0:1789762936.440173:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022194688:12345-0@lo:13:osp-pre-1-0.0 Request processed in 219us (688us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762936.440186:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022195200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.440191:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:1.0:1789762936.440212:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022194816:12345-0@lo:400:kworker.0 Request processed in 177us (604us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762936.440228:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 353 00000100:00100000:3.0:1789762936.440250:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022194944:0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.440297:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e902dc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3273:1876702022195200:0@lo:400:kworker.0 00000100:00100000:2.0:1789762936.440308:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3273:x1876702022195200:12345-0@lo:400:kworker.0 Request processed in 126us (585us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762936.440322:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022195328 00000100:00100000:2.0:1789762936.440332:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 371 00000100:00100000:1.0:1789762936.440333:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022195328 00000100:00100000:0.0:1789762936.440343:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8aa7bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022194688:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762936.440432:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:3.0:1789762936.440445:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e902de680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022195072:0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.440450:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022195328:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762936.440465:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762936.440476:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762936.440531:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8387f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022194816:0@lo:400:kworker.0 00000100:00100000:1.0:1789762936.440544:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022195328:12345-0@lo:400:kworker.0 Request processed in 95us (727us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762936.440557:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:2.0:1789762936.440579:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e902dfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022195328:0@lo:400:kworker.0 00000100:00100000:0.0:1789762936.474220:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459777938085 time 459778099748 state 0x0 ref 1 00000100:00100000:0.0:1789762936.474254:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015134720:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762936.474304:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0591e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.474328:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762936.474334:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762936.474343:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0591e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.474386:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0591e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.479557:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0591e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.479577:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0591e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.479589:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0591e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.479599:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762936.479605:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0591e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.479616:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0591e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.479637:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0591e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.479647:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0591e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762936.479737:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015134720:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5491us (195290us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762936.479750:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1293 00000100:00100000:0.0:1789762936.479775:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459777938085 time 459778099748 state 0x2 ref 0 00000100:00100000:3.0:1789762936.483696:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762936.483801:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015134848 00000100:00100000:1.0:1789762936.483818:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015134848 00000100:00100000:1.0:1789762936.483893:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459777938085 time 459778099748 state 0x0 ref 1 00000100:00100000:1.0:1789762936.483919:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459978099748 time 459778099748 state 0x1 ref 1 00000100:00100000:0.0:1789762936.674421:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459978099748 time 459978299962 state 0x0 ref 1 00000100:00100000:0.0:1789762936.674471:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015134848:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762936.674558:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05925 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.674590:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762936.674600:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762936.674612:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05925 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.674639:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05925 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.680840:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05925 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.680861:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05925 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.680875:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05925 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.680886:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762936.680896:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05925 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.680909:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05925 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.680933:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05925 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762936.680943:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05925 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762936.681028:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902de680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015134848:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6577us (197333us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762936.681042:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1294 00000100:00100000:0.0:1789762936.681057:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459978099748 time 459978299962 state 0x2 ref 0 00000100:00100000:2.0:1789762936.684888:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762936.684979:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015134976 00000100:00100000:0.0:1789762936.684995:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015134976 00000100:00100000:0.0:1789762936.685071:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 459978099748 time 459978299962 state 0x0 ref 1 00000100:00100000:0.0:1789762936.685136:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460178299962 time 459978299962 state 0x1 ref 1 00010000:00010000:1.0:1789762936.759412:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762936.874526:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460178299962 time 460178419395 state 0x0 ref 1 00000100:00100000:1.0:1789762936.874554:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015134976:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762936.874592:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0592c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.874607:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762936.874612:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762936.874616:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0592c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.874627:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0592c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.877471:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0592c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.877482:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0592c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.877493:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0592c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.877498:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762936.877501:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0592c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.877507:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0592c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.877520:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0592c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762936.877525:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0592c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762936.877577:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015134976:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3033us (192696us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762936.877588:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1295 00000100:00100000:1.0:1789762936.877598:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460178299962 time 460178419395 state 0x2 ref 0 00000100:00100000:3.0:1789762936.879544:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762936.879579:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015135104 00000100:00100000:1.0:1789762936.879587:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015135104 00000100:00100000:1.0:1789762936.879620:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460178299962 time 460178419395 state 0x0 ref 1 00000100:00100000:1.0:1789762936.879634:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460378419395 time 460178419395 state 0x1 ref 1 00000100:00100000:0.0:1789762937.074649:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460378419395 time 460378543215 state 0x0 ref 1 00000100:00100000:0.0:1789762937.074700:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015135104:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:1.0:1789762937.074741:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762937.074753:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05933 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.074768:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762937.074771:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762937.074776:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05933 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.074787:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05933 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.080123:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05933 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.080134:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05933 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.080142:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05933 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.080148:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762937.080152:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05933 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.080158:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05933 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.080177:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05933 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.080182:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05933 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762937.080229:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015135104:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5559us (200685us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762937.080235:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1296 00000100:00100000:0.0:1789762937.080243:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460378419395 time 460378543215 state 0x2 ref 0 00000100:00100000:3.0:1789762937.083498:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762937.083542:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015135232 00000100:00100000:1.0:1789762937.083549:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015135232 00000100:00100000:1.0:1789762937.083587:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460378419395 time 460378543215 state 0x0 ref 1 00000100:00100000:1.0:1789762937.083609:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460578543215 time 460378543215 state 0x1 ref 1 00000100:00100000:0.0:1789762937.274734:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460578543215 time 460578629957 state 0x0 ref 1 00000100:00100000:0.0:1789762937.274760:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015135232:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762937.274798:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0593a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.274812:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762937.274816:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762937.274820:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0593a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.274831:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0593a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.279213:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0593a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.279224:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0593a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.279231:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0593a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.279236:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762937.279239:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0593a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.279245:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0593a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.279258:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0593a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.279263:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0593a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762937.279334:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902de300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015135232:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4577us (195831us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762937.279358:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1297 00000100:00100000:0.0:1789762937.279366:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460578543215 time 460578629957 state 0x2 ref 0 00000100:00100000:2.0:1789762937.281589:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762937.281625:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015135360 00000100:00100000:0.0:1789762937.281633:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015135360 00000100:00100000:0.0:1789762937.281669:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460578543215 time 460578629957 state 0x0 ref 1 00000100:00100000:0.0:1789762937.281680:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460778629957 time 460578629957 state 0x1 ref 1 00000100:00100000:1.0:1789762937.474885:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460778629957 time 460778775335 state 0x0 ref 1 00000100:00100000:1.0:1789762937.474919:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015135360:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762937.474959:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05941 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.474976:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762937.474981:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762937.474988:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05941 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.475004:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05941 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.479082:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05941 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.479135:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05941 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.479146:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05941 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.479155:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762937.479160:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05941 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.479171:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05941 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.479192:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05941 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762937.479201:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05941 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762937.479267:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015135360:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4360us (197675us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762937.479278:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1298 00000100:00100000:1.0:1789762937.479290:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460778629957 time 460778775335 state 0x2 ref 0 00000100:00100000:3.0:1789762937.482510:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762937.482547:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015135488 00000100:00100000:1.0:1789762937.482556:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015135488 00000100:00100000:1.0:1789762937.482595:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460778629957 time 460778775335 state 0x0 ref 1 00000100:00100000:1.0:1789762937.482607:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460978775335 time 460778775335 state 0x1 ref 1 00000100:00100000:0.0:1789762937.674965:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460978775335 time 460978855688 state 0x0 ref 1 00000100:00100000:0.0:1789762937.674998:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015135488:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762937.675036:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05948 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.675051:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762937.675054:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762937.675059:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05948 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.675070:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05948 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.680038:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05948 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.680049:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05948 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.680057:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05948 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.680063:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762937.680066:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05948 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.680072:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05948 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.680101:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05948 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.680107:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05948 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762937.680157:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015135488:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5172us (197648us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762937.680164:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1299 00000100:00100000:0.0:1789762937.680171:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460978775335 time 460978855688 state 0x2 ref 0 00000100:00100000:3.0:1789762937.683210:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762937.683260:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015135616 00000100:00100000:1.0:1789762937.683269:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015135616 00000100:00100000:1.0:1789762937.683307:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 460978775335 time 460978855688 state 0x0 ref 1 00000100:00100000:1.0:1789762937.683318:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461178855688 time 460978855688 state 0x1 ref 1 00010000:00010000:3.0:1789762937.783231:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762937.875077:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461178855688 time 461178972462 state 0x0 ref 1 00000100:00100000:0.0:1789762937.875121:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015135616:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762937.875159:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0594f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.875172:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762937.875176:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762937.875180:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0594f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.875192:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0594f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.879235:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0594f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.879246:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0594f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.879254:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0594f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.879259:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762937.879263:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0594f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.879270:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0594f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.879289:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0594f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762937.879294:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0594f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762937.879354:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015135616:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4243us (196144us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762937.879362:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1300 00000100:00100000:0.0:1789762937.879370:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461178855688 time 461178972462 state 0x2 ref 0 00000100:00100000:3.0:1789762937.882306:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762937.882349:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015135744 00000100:00100000:0.0:1789762937.882356:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015135744 00000100:00100000:0.0:1789762937.882387:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461178855688 time 461178972462 state 0x0 ref 1 00000100:00100000:0.0:1789762937.882397:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461378972462 time 461178972462 state 0x1 ref 1 00000100:00100000:1.0:1789762938.075160:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461378972462 time 461379056860 state 0x0 ref 1 00000100:00100000:1.0:1789762938.075186:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015135744:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762938.075223:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05956 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.075237:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762938.075240:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762938.075244:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05956 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.075256:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05956 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.078720:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05956 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.078734:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05956 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.078744:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05956 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.078753:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762938.078758:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05956 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.078769:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05956 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.078787:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05956 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.078793:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05956 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.078849:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015135744:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3674us (196544us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762938.078856:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1301 00000100:00100000:1.0:1789762938.078863:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461378972462 time 461379056860 state 0x2 ref 0 00000100:00100000:0.0:1789762938.081241:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762938.081276:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015135872 00000100:00100000:1.0:1789762938.081284:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015135872 00000100:00100000:1.0:1789762938.081320:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461378972462 time 461379056860 state 0x0 ref 1 00000100:00100000:1.0:1789762938.081331:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461579056860 time 461379056860 state 0x1 ref 1 00000100:00100000:0.0:1789762938.275241:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461579056860 time 461579137198 state 0x0 ref 1 00000100:00100000:0.0:1789762938.275267:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015135872:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762938.275304:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0595d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.275323:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762938.275327:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762938.275347:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0595d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.275359:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0595d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.278190:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0595d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.278201:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0595d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.278208:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0595d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.278214:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762938.278218:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0595d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.278225:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0595d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.278239:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0595d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.278249:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b0595d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762938.278292:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015135872:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3036us (197053us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762938.278300:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1302 00000100:00100000:0.0:1789762938.278307:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461579056860 time 461579137198 state 0x2 ref 0 00000100:00100000:0.0:1789762938.280391:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762938.280445:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015136000 00000100:00100000:1.0:1789762938.280455:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015136000 00000100:00100000:1.0:1789762938.280492:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461579056860 time 461579137198 state 0x0 ref 1 00000100:00100000:1.0:1789762938.280503:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461779137198 time 461579137198 state 0x1 ref 1 00000100:00100000:0.0:1789762938.475359:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461779137198 time 461779239850 state 0x0 ref 1 00000100:00100000:0.0:1789762938.475385:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015136000:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762938.475435:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05964 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.475451:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762938.475456:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762938.475462:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05964 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.475477:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05964 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.478561:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05964 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.478582:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05964 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.478588:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05964 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.478594:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762938.478597:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05964 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.478603:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05964 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.478616:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05964 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762938.478621:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b9e91e5/0x416193e012b05964 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762938.478671:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015136000:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3294us (198281us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762938.478680:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1303 00000100:00100000:2.0:1789762938.478688:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461779137198 time 461779239850 state 0x2 ref 0 00000100:00100000:3.0:1789762938.480514:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762938.480569:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015136128 00000100:00100000:0.0:1789762938.480578:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015136128 00000100:00100000:0.0:1789762938.480616:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461779137198 time 461779239850 state 0x0 ref 1 00000100:00100000:0.0:1789762938.480626:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461979239850 time 461779239850 state 0x1 ref 1 00000100:00100000:1.0:1789762938.675459:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461979239850 time 461979355508 state 0x0 ref 1 00000100:00100000:1.0:1789762938.675484:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015136128:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762938.675522:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0596b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.675537:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762938.675540:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762938.675545:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0596b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.675556:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0596b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.678634:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0596b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.678650:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0596b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.678660:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0596b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.678666:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762938.678670:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0596b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.678677:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0596b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.678691:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0596b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762938.678697:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b0596b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762938.678744:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015136128:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3270us (198231us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762938.678751:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1304 00000100:00100000:1.0:1789762938.678764:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461979239850 time 461979355508 state 0x2 ref 0 00000100:00100000:3.0:1789762938.680647:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762938.680684:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015136256 00000100:00100000:1.0:1789762938.680691:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015136256 00000100:00100000:1.0:1789762938.680724:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 461979239850 time 461979355508 state 0x0 ref 1 00000100:00100000:1.0:1789762938.680734:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462179355508 time 461979355508 state 0x1 ref 1 00010000:00010000:0.0:1789762938.807204:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762938.875558:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462179355508 time 462179452783 state 0x0 ref 1 00000100:00100000:0.0:1789762938.875584:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015136256:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762938.875621:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d18152c4/0x416193e012b05972 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.875639:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762938.875644:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762938.875651:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d18152c4/0x416193e012b05972 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.875667:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d18152c4/0x416193e012b05972 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.878460:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d18152c4/0x416193e012b05972 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.878470:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d18152c4/0x416193e012b05972 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.878477:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d18152c4/0x416193e012b05972 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.878483:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe10c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762938.878487:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d18152c4/0x416193e012b05972 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.878493:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d18152c4/0x416193e012b05972 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.878505:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d18152c4/0x416193e012b05972 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762938.878510:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d18152c4/0x416193e012b05972 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762938.878560:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015136256:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2987us (197913us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762938.878568:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1305 00000100:00100000:0.0:1789762938.878576:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462179355508 time 462179452783 state 0x2 ref 0 00000100:00100000:0.0:1789762938.880502:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762938.880546:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015136384 00000100:00100000:1.0:1789762938.880554:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015136384 00000100:00100000:1.0:1789762938.880588:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462179355508 time 462179452783 state 0x0 ref 1 00000100:00100000:1.0:1789762938.880599:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462379452783 time 462179452783 state 0x1 ref 1 00000100:00100000:2.0:1789762939.075649:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462379452783 time 462379544945 state 0x0 ref 1 00000100:00100000:2.0:1789762939.075699:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015136384:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:1.0:1789762939.075724:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789762939.075737:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05979 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.075751:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762939.075756:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762939.075760:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05979 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.075771:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05979 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.079142:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05979 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.079153:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05979 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.079161:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05979 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.079167:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb5c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762939.079171:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05979 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.079187:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05979 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.079201:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05979 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.079206:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cbe3ac7c/0x416193e012b05979 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762939.079253:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015136384:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3576us (198752us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762939.079261:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1306 00000100:00100000:2.0:1789762939.079269:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462379452783 time 462379544945 state 0x2 ref 0 00000100:00100000:3.0:1789762939.081183:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762939.081256:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015136512 00000100:00100000:0.0:1789762939.081265:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015136512 00000100:00100000:0.0:1789762939.081300:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462379452783 time 462379544945 state 0x0 ref 1 00000100:00100000:0.0:1789762939.081311:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462579544945 time 462379544945 state 0x1 ref 1 00000100:00100000:1.0:1789762939.275749:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462579544945 time 462579639867 state 0x0 ref 1 00000100:00100000:1.0:1789762939.275783:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e902dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015136512:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762939.275827:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05980 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.275846:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762939.275851:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762939.275858:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05980 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.275873:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05980 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.279423:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05980 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.279436:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05980 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.279447:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05980 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.279456:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762939.279461:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05980 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.279471:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05980 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.279490:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05980 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.279500:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05980 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762939.279559:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e902dc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015136512:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3790us (198376us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762939.279570:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1307 00000100:00100000:1.0:1789762939.279581:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462579544945 time 462579639867 state 0x2 ref 0 00000100:00100000:0.0:1789762939.281591:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762939.281625:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015136640 00000100:00100000:1.0:1789762939.281632:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015136640 00000100:00100000:1.0:1789762939.281666:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462579544945 time 462579639867 state 0x0 ref 1 00000100:00100000:1.0:1789762939.281676:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462779639867 time 462579639867 state 0x1 ref 1 00000100:00100000:0.0:1789762939.475833:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462779639867 time 462779730198 state 0x0 ref 1 00000100:00100000:0.0:1789762939.475855:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015136640:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762939.475890:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d5d0b4a/0x416193e012b05987 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762939.475904:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762939.475908:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762939.475912:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d5d0b4a/0x416193e012b05987 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762939.475924:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d5d0b4a/0x416193e012b05987 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.478549:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d5d0b4a/0x416193e012b05987 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.478560:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d5d0b4a/0x416193e012b05987 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.478567:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d5d0b4a/0x416193e012b05987 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.478573:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e86d23a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762939.478577:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d5d0b4a/0x416193e012b05987 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.478584:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d5d0b4a/0x416193e012b05987 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.478599:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d5d0b4a/0x416193e012b05987 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762939.478605:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d5d0b4a/0x416193e012b05987 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762939.478653:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015136640:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2804us (197061us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762939.478659:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1308 00000100:00100000:3.0:1789762939.478666:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462779639867 time 462779730198 state 0x2 ref 0 00000100:00100000:3.0:1789762939.480734:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762939.480779:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015136768 00000100:00100000:1.0:1789762939.480788:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015136768 00000100:00100000:1.0:1789762939.480832:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462779639867 time 462779730198 state 0x0 ref 1 00000100:00100000:1.0:1789762939.480847:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462979730198 time 462779730198 state 0x1 ref 1 00000100:00100000:2.0:1789762939.675941:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462979730198 time 462979833207 state 0x0 ref 1 00000100:00100000:2.0:1789762939.675969:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015136768:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762939.676008:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0598e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.676025:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762939.676031:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762939.676037:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0598e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762939.676070:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0598e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762939.679268:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0598e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762939.679279:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0598e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762939.679286:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0598e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762939.679291:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfeb4200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762939.679295:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0598e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762939.679302:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0598e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762939.679315:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0598e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762939.679320:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e8859828/0x416193e012b0598e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762939.679382:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015136768:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3424us (198650us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762939.679390:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1309 00000100:00100000:0.0:1789762939.679398:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462979730198 time 462979833207 state 0x2 ref 0 00000100:00100000:3.0:1789762939.681260:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762939.681289:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015136896 00000100:00100000:0.0:1789762939.681296:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015136896 00000100:00100000:0.0:1789762939.681344:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 462979730198 time 462979833207 state 0x0 ref 1 00000100:00100000:0.0:1789762939.681355:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463179833207 time 462979833207 state 0x1 ref 1 00010000:00010000:0.0:1789762939.831185:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762939.876042:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463179833207 time 463179938378 state 0x0 ref 1 00000100:00100000:1.0:1789762939.876068:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8387e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015136896:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762939.876131:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05995 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.876146:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762939.876149:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762939.876154:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05995 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.876166:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05995 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.879122:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05995 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.879133:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05995 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.879140:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05995 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.879145:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762939.879149:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05995 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.879155:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05995 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.879168:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05995 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762939.879174:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b05995 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762939.879225:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8387e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015136896:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3167us (197964us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762939.879232:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1310 00000100:00100000:1.0:1789762939.879240:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463179833207 time 463179938378 state 0x2 ref 0 00000100:00100000:3.0:1789762939.881290:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762939.881338:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015137024 00000100:00100000:1.0:1789762939.881348:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015137024 00000100:00100000:1.0:1789762939.881390:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463179833207 time 463179938378 state 0x0 ref 1 00000100:00100000:1.0:1789762939.881401:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463379938378 time 463179938378 state 0x1 ref 1 00000100:00100000:3.0:1789762940.076134:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463379938378 time 463380031055 state 0x0 ref 1 00000100:00100000:3.0:1789762940.076157:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015137024:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762940.076205:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fa64281c/0x416193e012b0599c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.076220:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762940.076223:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762940.076228:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fa64281c/0x416193e012b0599c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.076238:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fa64281c/0x416193e012b0599c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.079109:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000fa64281c/0x416193e012b0599c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.079120:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fa64281c/0x416193e012b0599c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.079127:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fa64281c/0x416193e012b0599c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.079137:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe11a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762940.079140:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa64281c/0x416193e012b0599c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.079146:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fa64281c/0x416193e012b0599c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.079158:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fa64281c/0x416193e012b0599c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.079163:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fa64281c/0x416193e012b0599c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762940.079209:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015137024:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3059us (197919us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762940.079216:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1311 00000100:00100000:3.0:1789762940.079223:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463379938378 time 463380031055 state 0x2 ref 0 00000100:00100000:3.0:1789762940.081247:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762940.081296:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015137152 00000100:00100000:1.0:1789762940.081304:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015137152 00000100:00100000:1.0:1789762940.081362:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463379938378 time 463380031055 state 0x0 ref 1 00000100:00100000:1.0:1789762940.081373:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463580031055 time 463380031055 state 0x1 ref 1 00000100:00100000:0.0:1789762940.276229:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463580031055 time 463580124352 state 0x0 ref 1 00000100:00100000:0.0:1789762940.276258:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015137152:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762940.276294:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000be714456/0x416193e012b059a3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.276307:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762940.276311:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762940.276316:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000be714456/0x416193e012b059a3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.276340:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000be714456/0x416193e012b059a3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.279219:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000be714456/0x416193e012b059a3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.279229:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000be714456/0x416193e012b059a3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.279236:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000be714456/0x416193e012b059a3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.279241:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762940.279244:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000be714456/0x416193e012b059a3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.279254:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000be714456/0x416193e012b059a3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.279268:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000be714456/0x416193e012b059a3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.279274:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000be714456/0x416193e012b059a3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762940.279330:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015137152:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3071us (198070us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762940.279338:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1312 00000100:00100000:0.0:1789762940.279346:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463580031055 time 463580124352 state 0x2 ref 0 00000100:00100000:0.0:1789762940.281189:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762940.281229:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015137280 00000100:00100000:3.0:1789762940.281237:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015137280 00000100:00100000:3.0:1789762940.281274:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463580031055 time 463580124352 state 0x0 ref 1 00000100:00100000:3.0:1789762940.281285:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463780124352 time 463580124352 state 0x1 ref 1 00000100:00100000:1.0:1789762940.476414:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463780124352 time 463780310862 state 0x0 ref 1 00000100:00100000:1.0:1789762940.476435:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015137280:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762940.476472:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b059aa lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.476487:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.476491:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762940.476495:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b059aa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.476506:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b059aa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.480844:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b059aa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.480857:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b059aa lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.480868:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b059aa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.480877:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762940.480882:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b059aa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.480916:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b059aa lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.480949:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b059aa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762940.480984:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2f13f0b/0x416193e012b059aa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762940.481040:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015137280:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4610us (199849us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762940.481050:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1313 00000100:00100000:1.0:1789762940.481061:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463780124352 time 463780310862 state 0x2 ref 0 00000100:00100000:2.0:1789762940.483698:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762940.483738:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015137408 00000100:00100000:1.0:1789762940.483746:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015137408 00000100:00100000:1.0:1789762940.483782:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463780124352 time 463780310862 state 0x0 ref 1 00000100:00100000:1.0:1789762940.483793:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463980310862 time 463780310862 state 0x1 ref 1 00000100:00100000:3.0:1789762940.676526:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463980310862 time 463980418462 state 0x0 ref 1 00000100:00100000:3.0:1789762940.676557:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015137408:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762940.676598:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b059b1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.676616:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762940.676621:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762940.676627:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b059b1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762940.676642:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b059b1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.680144:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b059b1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.680166:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b059b1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.680173:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b059b1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.680179:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762940.680184:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b059b1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.680193:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b059b1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.680206:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b059b1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.680212:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b059b1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762940.680282:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015137408:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3738us (196584us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762940.680290:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1314 00000100:00100000:0.0:1789762940.680302:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463980310862 time 463980418462 state 0x2 ref 0 00000100:00100000:2.0:1789762940.682610:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762940.682665:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015137536 00000100:00100000:1.0:1789762940.682673:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015137536 00000100:00100000:1.0:1789762940.682710:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 463980310862 time 463980418462 state 0x0 ref 1 00000100:00100000:1.0:1789762940.682722:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464180418462 time 463980418462 state 0x1 ref 1 00010000:00010000:1.0:1789762940.855229:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762940.876597:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464180418462 time 464180493849 state 0x0 ref 1 00000100:00100000:0.0:1789762940.876623:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015137536:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762940.876659:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059b8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.876674:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762940.876678:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762940.876682:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059b8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.876693:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059b8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.880142:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059b8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.880152:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059b8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.880159:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059b8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.880164:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762940.880168:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059b8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.880174:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059b8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.880187:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059b8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762940.880192:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059b8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762940.880247:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa79180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015137536:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3634us (197646us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762940.880256:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1315 00000100:00100000:0.0:1789762940.880267:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464180418462 time 464180493849 state 0x2 ref 0 00000100:00100000:2.0:1789762940.882982:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762940.883026:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015137664 00000100:00100000:3.0:1789762940.883036:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015137664 00000100:00100000:3.0:1789762940.883073:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464180418462 time 464180493849 state 0x0 ref 1 00000100:00100000:3.0:1789762940.883083:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464380493849 time 464180493849 state 0x1 ref 1 00000100:00100000:3.0:1789762941.026269:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762941.026303:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762941.026328:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015137920 00000100:00100000:2.0:1789762941.026340:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015137920 00000100:00100000:0.0:1789762941.026373:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015137792 00000100:00100000:3.0:1789762941.026373:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762941.026381:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015137792 00000100:00080000:1.0:1789762941.026414:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762941.026421:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015138048 00000100:00100000:2.0:1789762941.026426:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015138048 00000100:00100000:0.0:1789762941.026427:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 354 00000100:00100000:0.0:1789762941.026441:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015137792:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:1.0:1789762941.026446:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762941.026465:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 372 00000100:00100000:2.0:1789762941.026476:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 373 00000100:00100000:3.0:1789762941.026481:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702015137920:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762941.026492:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015138048:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762941.026504:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015137792:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 67us (198us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762941.026511:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 354 00000100:00100000:3.0:1789762941.026525:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702015137920:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 50us (261us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762941.026532:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015138048:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 45us (158us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762941.026533:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 372 00000100:00100000:2.0:1789762941.026546:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 373 00000100:00100000:2.0:1789762941.047232:0:3272:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789762941.047254:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb18a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022195456:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762941.047290:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762941.047343:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022195456 00000100:00100000:0.0:1789762941.047349:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022195456 00000100:00080000:1.0:1789762941.047375:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762941.047389:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:0.0:1789762941.047397:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022195456:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762941.047421:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762941.047460:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022195456:12345-0@lo:13:osp-pre-0-0.0 Request processed in 64us (169us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762941.047465:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:2.0:1789762941.047499:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb18a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022195456:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762941.076670:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464380493849 time 464380568172 state 0x0 ref 1 00000100:00100000:1.0:1789762941.076702:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015137664:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762941.076732:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cd4bb8c7/0x416193e012b059bf lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762941.076746:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.076750:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762941.076762:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd4bb8c7/0x416193e012b059bf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762941.076762:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789762941.076781:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd4bb8c7/0x416193e012b059bf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.079798:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cd4bb8c7/0x416193e012b059bf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.079809:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd4bb8c7/0x416193e012b059bf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.079830:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd4bb8c7/0x416193e012b059bf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.079836:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f08a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762941.079840:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd4bb8c7/0x416193e012b059bf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.079846:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd4bb8c7/0x416193e012b059bf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.079860:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd4bb8c7/0x416193e012b059bf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.079866:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd4bb8c7/0x416193e012b059bf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762941.079913:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015137664:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3229us (196932us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762941.079920:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1316 00000100:00100000:2.0:1789762941.079927:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464380493849 time 464380568172 state 0x2 ref 0 00000100:00100000:3.0:1789762941.081897:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762941.081944:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015138176 00000100:00100000:1.0:1789762941.081950:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015138176 00000100:00100000:1.0:1789762941.081981:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464380493849 time 464380568172 state 0x0 ref 1 00000100:00100000:1.0:1789762941.081991:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464580568172 time 464380568172 state 0x1 ref 1 00000100:00100000:0.0:1789762941.276808:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464580568172 time 464580704714 state 0x0 ref 1 00000100:00100000:0.0:1789762941.276834:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015138176:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762941.276884:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059c6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.276898:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762941.276902:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762941.276906:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059c6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.276918:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059c6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.280434:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059c6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.280445:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059c6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.280452:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059c6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.280457:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762941.280460:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059c6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.280466:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059c6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.280484:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059c6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.280489:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059c6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762941.280541:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015138176:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3716us (198644us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762941.280551:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1317 00000100:00100000:0.0:1789762941.280561:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464580568172 time 464580704714 state 0x2 ref 0 00000100:00100000:0.0:1789762941.282896:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762941.282947:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015138304 00000100:00100000:1.0:1789762941.282954:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015138304 00000100:00100000:1.0:1789762941.282988:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464580568172 time 464580704714 state 0x0 ref 1 00000100:00100000:1.0:1789762941.283000:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464780704714 time 464580704714 state 0x1 ref 1 00000100:00100000:0.0:1789762941.476897:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464780704714 time 464780793640 state 0x0 ref 1 00000100:00100000:0.0:1789762941.476918:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015138304:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762941.476953:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059cd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.476967:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762941.476970:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762941.476975:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059cd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.476986:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059cd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.479663:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059cd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.479677:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059cd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.479687:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059cd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.479694:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762941.479699:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059cd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.479710:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059cd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.479728:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059cd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.479736:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059cd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762941.479785:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015138304:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2874us (196890us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762941.479793:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1318 00000100:00100000:0.0:1789762941.479802:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464780704714 time 464780793640 state 0x2 ref 0 00000100:00100000:0.0:1789762941.481700:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762941.481732:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015138432 00000100:00100000:3.0:1789762941.481740:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015138432 00000100:00100000:3.0:1789762941.481774:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464780704714 time 464780793640 state 0x0 ref 1 00000100:00100000:3.0:1789762941.481785:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464980793640 time 464780793640 state 0x1 ref 1 00000100:00100000:2.0:1789762941.559282:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022195840:0@lo:400:kworker.0 00000100:00100000:3.0:1789762941.559282:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022195584:0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.559291:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022195968:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.559324:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8aa7aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022195712:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762941.559384:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762941.559392:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762941.559399:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762941.559423:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022195840 00000100:00100000:1.0:1789762941.559430:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022195840 00000100:00100000:0.0:1789762941.559454:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022196224:0@lo:400:kworker.0 00000100:00100000:3.0:1789762941.559461:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022196096:0@lo:400:kworker.0 00000100:00100000:2.0:1789762941.559482:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022195712 00000100:00100000:1.0:1789762941.559489:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:0.0:1789762941.559490:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762941.559499:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762941.559501:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022195712 00000100:00100000:1.0:1789762941.559504:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022195840:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762941.559554:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022195584 00000100:00100000:2.0:1789762941.559555:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 156 00000100:00100000:0.0:1789762941.559559:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022196096 00000100:00100000:3.0:1789762941.559560:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022195584 00000100:00100000:0.0:1789762941.559566:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022196096 00000100:00100000:2.0:1789762941.559566:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022195712:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762941.559574:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3274:x1876702022195840:12345-0@lo:400:kworker.0 Request processed in 72us (187us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762941.559585:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00002000:00000024:2.0:1789762941.559597:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762941.559607:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:1.0:1789762941.559612:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 375 00000100:00100000:3.0:1789762941.559620:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022195584:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.559626:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022196096:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.559647:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762941.559661:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022195712:12345-0@lo:13:osp-pre-1-0.0 Request processed in 97us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762941.559668:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 156 00000100:00100000:3.0:1789762941.559697:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022195584:12345-0@lo:400:kworker.0 Request processed in 78us (305us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762941.559698:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022196096:12345-0@lo:400:kworker.0 Request processed in 75us (198us total) trans 0 rc 0/0 00000100:00080000:2.0:1789762941.559699:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789762941.559705:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762941.559721:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8aa7aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022195712:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762941.559722:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00100000:1.0:1789762941.559723:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 375 00000100:00100000:2.0:1789762941.559728:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3274:1876702022195840:0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.559751:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022195968 00000100:00100000:1.0:1789762941.559756:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022195968 00000100:00100000:2.0:1789762941.559786:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022196096:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.559787:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022196224 00000100:00100000:0.0:1789762941.559792:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022196224 00000100:00100000:1.0:1789762941.559794:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:1.0:1789762941.559802:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022195968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762941.559805:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022195584:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.559833:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:0.0:1789762941.559844:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022196224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762941.559852:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3271:x1876702022195968:12345-0@lo:400:kworker.0 Request processed in 52us (213us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762941.559854:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762941.559863:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00100000:0.0:1789762941.559895:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022196224:12345-0@lo:400:kworker.0 Request processed in 53us (404us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762941.559896:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3271:1876702022195968:0@lo:400:kworker.0 00000100:00100000:0.0:1789762941.559912:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00100000:2.0:1789762941.559923:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022196224:0@lo:400:kworker.0 00000100:00100000:2.0:1789762941.677012:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464980793640 time 464980906097 state 0x0 ref 1 00000100:00100000:2.0:1789762941.677045:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015138432:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762941.677082:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059d4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.677121:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762941.677125:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762941.677130:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059d4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.677141:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059d4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.680244:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059d4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.680255:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059d4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.680262:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059d4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.680267:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762941.680270:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059d4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.680277:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059d4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.680289:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059d4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762941.680294:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059d4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762941.680355:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015138432:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3327us (198655us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762941.680362:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1319 00000100:00100000:2.0:1789762941.680375:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464980793640 time 464980906097 state 0x2 ref 0 00000100:00100000:3.0:1789762941.682199:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762941.682243:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015138560 00000100:00100000:1.0:1789762941.682251:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015138560 00000100:00100000:1.0:1789762941.682287:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 464980793640 time 464980906097 state 0x0 ref 1 00000100:00100000:1.0:1789762941.682298:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465180906097 time 464980906097 state 0x1 ref 1 00000100:00100000:0.0:1789762941.877120:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465180906097 time 465180991514 state 0x0 ref 1 00000100:00100000:0.0:1789762941.877147:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015138560:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762941.877182:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059db lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.877197:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762941.877200:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762941.877206:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059db lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.877218:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059db lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762941.879211:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762941.880064:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059db lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.880074:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059db lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.880082:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059db lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.880104:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762941.880108:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059db lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.880116:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059db lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.880131:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059db lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762941.880136:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059db lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762941.880190:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015138560:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3054us (197991us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762941.880198:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1320 00000100:00100000:0.0:1789762941.880205:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465180906097 time 465180991514 state 0x2 ref 0 00000100:00100000:3.0:1789762941.882053:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762941.882107:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015138688 00000100:00100000:1.0:1789762941.882115:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015138688 00000100:00100000:1.0:1789762941.882151:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465180906097 time 465180991514 state 0x0 ref 1 00000100:00100000:1.0:1789762941.882162:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465380991514 time 465180991514 state 0x1 ref 1 00000100:00100000:0.0:1789762942.077157:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465380991514 time 465381055965 state 0x0 ref 1 00000100:00100000:0.0:1789762942.077177:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015138688:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762942.077211:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059e2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.077225:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762942.077229:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762942.077233:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059e2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.077244:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059e2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.080023:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059e2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.080033:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059e2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.080043:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059e2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.080048:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762942.080052:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059e2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.080058:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059e2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.080070:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059e2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.080075:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059e2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762942.080140:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015138688:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2968us (198089us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762942.080147:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1321 00000100:00100000:0.0:1789762942.080155:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465380991514 time 465381055965 state 0x2 ref 0 00000100:00100000:3.0:1789762942.082408:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762942.082443:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015138816 00000100:00100000:0.0:1789762942.082450:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015138816 00000100:00100000:0.0:1789762942.082487:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465380991514 time 465381055965 state 0x0 ref 1 00000100:00100000:0.0:1789762942.082501:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465581055965 time 465381055965 state 0x1 ref 1 00000100:00100000:2.0:1789762942.277272:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465581055965 time 465581167895 state 0x0 ref 1 00000100:00100000:2.0:1789762942.277298:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015138816:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762942.277351:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059e9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762942.277365:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762942.277368:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762942.277373:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059e9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762942.277384:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059e9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762942.281103:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059e9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762942.281115:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059e9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762942.281122:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059e9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762942.281128:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762942.281143:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059e9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762942.281151:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059e9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762942.281164:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059e9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762942.281170:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059e9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762942.281218:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015138816:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3930us (198811us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762942.281225:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1322 00000100:00100000:2.0:1789762942.281233:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465581055965 time 465581167895 state 0x2 ref 0 00000100:00100000:3.0:1789762942.283177:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762942.283225:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015138944 00000100:00100000:1.0:1789762942.283234:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015138944 00000100:00100000:1.0:1789762942.283272:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465581055965 time 465581167895 state 0x0 ref 1 00000100:00100000:1.0:1789762942.283285:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465781167895 time 465581167895 state 0x1 ref 1 00000100:00100000:0.0:1789762942.477370:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465781167895 time 465781266400 state 0x0 ref 1 00000100:00100000:0.0:1789762942.477393:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015138944:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762942.477429:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059f0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.477443:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762942.477447:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762942.477451:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059f0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.477463:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059f0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.480544:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059f0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.480555:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059f0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.480562:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059f0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.480567:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762942.480570:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059f0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.480576:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059f0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.480588:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059f0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762942.480594:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005d96e14e/0x416193e012b059f0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762942.480642:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015138944:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3256us (197465us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762942.480651:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1323 00000100:00100000:3.0:1789762942.480659:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465781167895 time 465781266400 state 0x2 ref 0 00000100:00100000:3.0:1789762942.482490:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762942.482541:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015139072 00000100:00100000:1.0:1789762942.482549:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015139072 00000100:00100000:1.0:1789762942.482584:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465781167895 time 465781266400 state 0x0 ref 1 00000100:00100000:1.0:1789762942.482595:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465981266400 time 465781266400 state 0x1 ref 1 00000100:00100000:0.0:1789762942.677482:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465981266400 time 465981378500 state 0x0 ref 1 00000100:00100000:0.0:1789762942.677508:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015139072:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762942.677545:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b059f7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.677560:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762942.677563:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762942.677568:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b059f7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.677580:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b059f7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.680709:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b059f7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.680721:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b059f7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.680728:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b059f7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.680734:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762942.680737:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b059f7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.680750:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b059f7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.680764:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b059f7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762942.680769:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b059f7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762942.680822:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015139072:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3323us (198333us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762942.680829:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1324 00000100:00100000:0.0:1789762942.680837:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465981266400 time 465981378500 state 0x2 ref 0 00000100:00100000:3.0:1789762942.683043:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762942.683081:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015139200 00000100:00100000:0.0:1789762942.683109:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015139200 00000100:00100000:0.0:1789762942.683151:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 465981266400 time 465981378500 state 0x0 ref 1 00000100:00100000:0.0:1789762942.683166:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466181378500 time 465981378500 state 0x1 ref 1 00000100:00100000:2.0:1789762942.877841:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466181378500 time 466181718527 state 0x0 ref 1 00000100:00100000:2.0:1789762942.879171:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015139200:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762942.879234:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059fe lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762942.879252:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762942.879256:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762942.879263:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059fe lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762942.879296:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059fe lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.883407:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059fe lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.883421:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059fe lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.883428:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059fe lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.883433:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d865a6c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762942.883436:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059fe lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.883442:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059fe lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.883456:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059fe lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762942.883462:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000024c63e99/0x416193e012b059fe lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762942.883526:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015139200:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4375us (200480us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762942.883537:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1325 00000100:00100000:1.0:1789762942.883548:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466181378500 time 466181718527 state 0x2 ref 0 00000100:00100000:3.0:1789762942.885964:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762942.886017:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015139328 00000100:00100000:1.0:1789762942.886028:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015139328 00000100:00100000:1.0:1789762942.886121:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466181378500 time 466181718527 state 0x0 ref 1 00000100:00100000:1.0:1789762942.886137:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466381718527 time 466181718527 state 0x1 ref 1 00010000:00010000:0.0:1789762942.903194:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762943.077955:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466381718527 time 466381845027 state 0x0 ref 1 00000100:00100000:3.0:1789762943.078005:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015139328:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:2.0:1789762943.078043:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762943.078063:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000067433d16/0x416193e012b05a05 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.078082:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.078113:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762943.078120:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067433d16/0x416193e012b05a05 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.078134:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067433d16/0x416193e012b05a05 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.081915:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000067433d16/0x416193e012b05a05 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.081931:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067433d16/0x416193e012b05a05 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.081943:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067433d16/0x416193e012b05a05 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.081953:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe78c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762943.081969:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067433d16/0x416193e012b05a05 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.081986:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067433d16/0x416193e012b05a05 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.082005:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067433d16/0x416193e012b05a05 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.082012:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067433d16/0x416193e012b05a05 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762943.082207:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015139328:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4225us (196242us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762943.082219:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1326 00000100:00100000:0.0:1789762943.082241:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466381718527 time 466381845027 state 0x2 ref 0 00000100:00100000:3.0:1789762943.084493:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762943.084567:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015139456 00000100:00100000:1.0:1789762943.084576:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015139456 00000100:00100000:1.0:1789762943.084619:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466381718527 time 466381845027 state 0x0 ref 1 00000100:00100000:1.0:1789762943.084630:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466581845027 time 466381845027 state 0x1 ref 1 00000100:00100000:0.0:1789762943.278032:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466581845027 time 466581927475 state 0x0 ref 1 00000100:00100000:0.0:1789762943.278059:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015139456:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762943.278128:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a0c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.278143:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762943.278146:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762943.278151:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a0c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.278167:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a0c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.283140:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a0c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.283150:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a0c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.283157:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a0c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.283162:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762943.283166:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a0c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.283172:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a0c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.283186:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a0c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.283191:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a0c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762943.283242:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015139456:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5194us (198749us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762943.283250:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1327 00000100:00100000:0.0:1789762943.283259:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466581845027 time 466581927475 state 0x2 ref 0 00000100:00100000:1.0:1789762943.285168:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762943.285200:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015139584 00000100:00100000:0.0:1789762943.285207:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015139584 00000100:00100000:0.0:1789762943.285243:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466581845027 time 466581927475 state 0x0 ref 1 00000100:00100000:0.0:1789762943.285253:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466781927475 time 466581927475 state 0x1 ref 1 00000100:00100000:1.0:1789762943.478138:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466781927475 time 466782034044 state 0x0 ref 1 00000100:00100000:1.0:1789762943.478159:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015139584:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762943.478195:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000084d83cd/0x416193e012b05a13 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.478209:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762943.478212:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762943.478217:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000084d83cd/0x416193e012b05a13 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762943.478228:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000084d83cd/0x416193e012b05a13 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.481079:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000084d83cd/0x416193e012b05a13 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.481118:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000084d83cd/0x416193e012b05a13 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.481128:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000084d83cd/0x416193e012b05a13 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.481137:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfb2d400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762943.481142:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000084d83cd/0x416193e012b05a13 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.481151:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000084d83cd/0x416193e012b05a13 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.481168:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000084d83cd/0x416193e012b05a13 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762943.481177:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000084d83cd/0x416193e012b05a13 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762943.481250:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015139584:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3097us (196082us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762943.481260:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1328 00000100:00100000:2.0:1789762943.481270:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466781927475 time 466782034044 state 0x2 ref 0 00000100:00100000:3.0:1789762943.483100:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762943.483146:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015139712 00000100:00100000:1.0:1789762943.483155:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015139712 00000100:00100000:1.0:1789762943.483191:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466781927475 time 466782034044 state 0x0 ref 1 00000100:00100000:1.0:1789762943.483201:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466982034044 time 466782034044 state 0x1 ref 1 00000100:00100000:0.0:1789762943.678255:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466982034044 time 466982139372 state 0x0 ref 1 00000100:00100000:0.0:1789762943.678280:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8a6b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015139712:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762943.678334:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a1a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.678359:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762943.678362:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762943.678367:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a1a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.678378:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a1a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.681359:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a1a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.681374:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a1a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.681381:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a1a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.681386:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762943.681389:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a1a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.681395:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a1a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.681409:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a1a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762943.681415:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001bff94e7/0x416193e012b05a1a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762943.681464:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8a6b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015139712:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3192us (198377us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762943.681472:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1329 00000100:00100000:3.0:1789762943.681479:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466982034044 time 466982139372 state 0x2 ref 0 00000100:00100000:3.0:1789762943.683456:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762943.683505:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015139840 00000100:00100000:1.0:1789762943.683514:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015139840 00000100:00100000:1.0:1789762943.683551:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 466982034044 time 466982139372 state 0x0 ref 1 00000100:00100000:1.0:1789762943.683562:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467182139372 time 466982139372 state 0x1 ref 1 00000100:00100000:0.0:1789762943.878341:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467182139372 time 467182235724 state 0x0 ref 1 00000100:00100000:0.0:1789762943.878367:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015139840:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762943.878406:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a21 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.878419:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762943.878422:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762943.878427:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a21 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.878440:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a21 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.881797:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a21 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.881811:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a21 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.881822:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a21 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.881830:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762943.881835:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a21 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.881844:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a21 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.881862:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a21 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762943.881872:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a21 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762943.881928:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015139840:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3570us (198471us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762943.881935:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1330 00000100:00100000:0.0:1789762943.881943:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467182139372 time 467182235724 state 0x2 ref 0 00000100:00100000:1.0:1789762943.883856:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762943.883898:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015139968 00000100:00100000:0.0:1789762943.883905:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015139968 00000100:00100000:0.0:1789762943.883936:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467182139372 time 467182235724 state 0x0 ref 1 00000100:00100000:0.0:1789762943.883947:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467382235724 time 467182235724 state 0x1 ref 1 00010000:00010000:0.0:1789762943.927192:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762944.078428:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467382235724 time 467382323039 state 0x0 ref 1 00000100:00100000:2.0:1789762944.078453:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015139968:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762944.078491:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a28 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.078505:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762944.078508:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762944.078512:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a28 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.078523:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a28 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.081582:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a28 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.081592:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a28 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.081616:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a28 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.081621:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762944.081625:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a28 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.081631:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a28 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.081644:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a28 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.081650:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a28 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762944.081698:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015139968:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3254us (197841us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762944.081705:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1331 00000100:00100000:2.0:1789762944.081713:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467382235724 time 467382323039 state 0x2 ref 0 00000100:00100000:3.0:1789762944.083680:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762944.083726:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015140096 00000100:00100000:1.0:1789762944.083735:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015140096 00000100:00100000:1.0:1789762944.083772:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467382235724 time 467382323039 state 0x0 ref 1 00000100:00100000:1.0:1789762944.083783:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467582323039 time 467382323039 state 0x1 ref 1 00000100:00100000:3.0:1789762944.278627:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467582323039 time 467582506815 state 0x0 ref 1 00000100:00100000:3.0:1789762944.278675:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015140096:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762944.278740:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1b003a5/0x416193e012b05a2f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.278769:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762944.278777:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762944.278805:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1b003a5/0x416193e012b05a2f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.278820:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1b003a5/0x416193e012b05a2f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.282598:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1b003a5/0x416193e012b05a2f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.282612:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c1b003a5/0x416193e012b05a2f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.282620:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c1b003a5/0x416193e012b05a2f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.282626:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe79600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762944.282630:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1b003a5/0x416193e012b05a2f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.282638:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1b003a5/0x416193e012b05a2f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.282653:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c1b003a5/0x416193e012b05a2f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.282658:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1b003a5/0x416193e012b05a2f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762944.282716:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015140096:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4064us (199036us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762944.282726:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1332 00000100:00100000:3.0:1789762944.282735:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467582323039 time 467582506815 state 0x2 ref 0 00000100:00100000:3.0:1789762944.285074:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762944.285127:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015140224 00000100:00100000:1.0:1789762944.285135:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015140224 00000100:00100000:1.0:1789762944.285176:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467582323039 time 467582506815 state 0x0 ref 1 00000100:00100000:1.0:1789762944.285208:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467782506815 time 467582506815 state 0x1 ref 1 00000100:00100000:0.0:1789762944.478685:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467782506815 time 467782581032 state 0x0 ref 1 00000100:00100000:0.0:1789762944.478710:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015140224:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762944.478747:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a36 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762944.478760:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762944.478764:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762944.478769:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a36 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762944.478780:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a36 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762944.481913:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a36 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762944.481923:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a36 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762944.481930:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a36 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762944.481935:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762944.481939:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a36 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762944.481949:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a36 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762944.481962:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a36 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762944.481967:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a36 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762944.482016:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015140224:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3313us (196943us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762944.482023:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1333 00000100:00100000:0.0:1789762944.482031:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467782506815 time 467782581032 state 0x2 ref 0 00000100:00100000:1.0:1789762944.483854:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762944.483896:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015140352 00000100:00100000:0.0:1789762944.483906:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015140352 00000100:00100000:0.0:1789762944.483948:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467782506815 time 467782581032 state 0x0 ref 1 00000100:00100000:0.0:1789762944.483960:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467982581032 time 467782581032 state 0x1 ref 1 00000100:00100000:2.0:1789762944.678792:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467982581032 time 467982687885 state 0x0 ref 1 00000100:00100000:2.0:1789762944.678818:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015140352:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762944.678868:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a3d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.678881:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762944.678885:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762944.678889:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a3d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.678900:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a3d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.682349:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a3d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.682360:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a3d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.682368:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a3d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.682374:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762944.682378:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a3d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.682384:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a3d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.682416:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a3d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762944.682422:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a3d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762944.682473:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015140352:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3667us (198620us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762944.682481:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1334 00000100:00100000:2.0:1789762944.682489:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467982581032 time 467982687885 state 0x2 ref 0 00000100:00100000:3.0:1789762944.684340:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762944.684400:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015140480 00000100:00100000:1.0:1789762944.684411:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015140480 00000100:00100000:1.0:1789762944.684453:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 467982581032 time 467982687885 state 0x0 ref 1 00000100:00100000:1.0:1789762944.684463:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468182687885 time 467982687885 state 0x1 ref 1 00000100:00100000:3.0:1789762944.878876:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468182687885 time 468182771232 state 0x0 ref 1 00000100:00100000:3.0:1789762944.878899:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015140480:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762944.878936:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c71cf13e/0x416193e012b05a44 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.878949:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762944.878954:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762944.878959:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c71cf13e/0x416193e012b05a44 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.878972:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c71cf13e/0x416193e012b05a44 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.882032:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c71cf13e/0x416193e012b05a44 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.882048:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c71cf13e/0x416193e012b05a44 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.882058:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c71cf13e/0x416193e012b05a44 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.882066:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe78200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762944.882071:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c71cf13e/0x416193e012b05a44 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.882079:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c71cf13e/0x416193e012b05a44 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.882115:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c71cf13e/0x416193e012b05a44 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762944.882121:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c71cf13e/0x416193e012b05a44 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762944.882186:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015140480:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3291us (197844us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762944.882197:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1335 00000100:00100000:3.0:1789762944.882209:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468182687885 time 468182771232 state 0x2 ref 0 00000100:00100000:3.0:1789762944.884261:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762944.884312:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015140608 00000100:00100000:1.0:1789762944.884326:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015140608 00000100:00100000:1.0:1789762944.884366:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468182687885 time 468182771232 state 0x0 ref 1 00000100:00100000:1.0:1789762944.884378:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468382771232 time 468182771232 state 0x1 ref 1 00010000:00010000:1.0:1789762944.951194:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762945.078959:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468382771232 time 468382855109 state 0x0 ref 1 00000100:00100000:0.0:1789762945.079000:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015140608:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762945.079038:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a4b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762945.079039:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762945.079054:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762945.079058:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762945.079063:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a4b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.079075:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a4b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.082366:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a4b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.082381:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a4b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.082392:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a4b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.082399:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762945.082403:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a4b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.082410:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a4b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.082424:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a4b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.082434:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a4b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762945.082485:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015140608:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3511us (198226us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762945.082492:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1336 00000100:00100000:0.0:1789762945.082499:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468382771232 time 468382855109 state 0x2 ref 0 00000100:00100000:1.0:1789762945.084610:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762945.084649:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015140736 00000100:00100000:0.0:1789762945.084657:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015140736 00000100:00100000:0.0:1789762945.084692:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468382771232 time 468382855109 state 0x0 ref 1 00000100:00100000:0.0:1789762945.084702:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468582855109 time 468382855109 state 0x1 ref 1 00000100:00100000:2.0:1789762945.279114:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468582855109 time 468582992395 state 0x0 ref 1 00000100:00100000:2.0:1789762945.279140:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015140736:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762945.279177:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a52 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.279191:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762945.279195:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762945.279200:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a52 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.279211:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a52 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.282137:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a52 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.282147:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a52 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.282154:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a52 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.282160:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762945.282163:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a52 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.282169:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a52 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.282192:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a52 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.282197:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a52 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762945.282246:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015140736:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3116us (197637us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762945.282253:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1337 00000100:00100000:2.0:1789762945.282267:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468582855109 time 468582992395 state 0x2 ref 0 00000100:00100000:3.0:1789762945.284689:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762945.284749:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015140864 00000100:00100000:1.0:1789762945.284759:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015140864 00000100:00100000:1.0:1789762945.284807:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468582855109 time 468582992395 state 0x0 ref 1 00000100:00100000:1.0:1789762945.284818:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468782992395 time 468582992395 state 0x1 ref 1 00000100:00100000:3.0:1789762945.479211:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468782992395 time 468783105937 state 0x0 ref 1 00000100:00100000:3.0:1789762945.479232:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015140864:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762945.479301:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fe5ba83/0x416193e012b05a59 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.479315:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762945.479318:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762945.479323:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001fe5ba83/0x416193e012b05a59 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.479335:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001fe5ba83/0x416193e012b05a59 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.482442:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001fe5ba83/0x416193e012b05a59 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.482455:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001fe5ba83/0x416193e012b05a59 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.482461:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001fe5ba83/0x416193e012b05a59 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.482467:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe79a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762945.482471:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001fe5ba83/0x416193e012b05a59 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.482477:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001fe5ba83/0x416193e012b05a59 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.482491:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001fe5ba83/0x416193e012b05a59 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762945.482496:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001fe5ba83/0x416193e012b05a59 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762945.482541:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015140864:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3316us (197854us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762945.482549:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1338 00000100:00100000:3.0:1789762945.482556:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468782992395 time 468783105937 state 0x2 ref 0 00000100:00100000:3.0:1789762945.484527:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762945.484568:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015140992 00000100:00100000:1.0:1789762945.484577:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015140992 00000100:00100000:1.0:1789762945.484613:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468782992395 time 468783105937 state 0x0 ref 1 00000100:00100000:1.0:1789762945.484624:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468983105937 time 468783105937 state 0x1 ref 1 00000100:00100000:0.0:1789762945.679301:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468983105937 time 468983181442 state 0x0 ref 1 00000100:00100000:0.0:1789762945.679340:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015140992:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762945.679379:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a60 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.679392:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762945.679396:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762945.679401:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a60 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.679418:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a60 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.683046:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a60 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.683058:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a60 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.683065:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a60 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.683070:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762945.683074:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a60 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.683080:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a60 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.683110:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a60 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762945.683117:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a60 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762945.683167:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015140992:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3850us (198640us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762945.683175:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1339 00000100:00100000:0.0:1789762945.683182:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468983105937 time 468983181442 state 0x2 ref 0 00000100:00100000:1.0:1789762945.685844:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762945.685884:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015141120 00000100:00100000:0.0:1789762945.685894:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015141120 00000100:00100000:0.0:1789762945.685932:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 468983105937 time 468983181442 state 0x0 ref 1 00000100:00100000:0.0:1789762945.685942:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469183181442 time 468983181442 state 0x1 ref 1 00000100:00100000:2.0:1789762945.879391:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469183181442 time 469183286605 state 0x0 ref 1 00000100:00100000:2.0:1789762945.879417:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015141120:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762945.879454:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a67 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.879468:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762945.879471:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762945.879476:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a67 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.879490:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a67 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.883465:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a67 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.883479:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a67 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.883490:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a67 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.883499:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762945.883518:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a67 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.883529:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a67 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.883545:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a67 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762945.883550:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05a67 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762945.883622:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015141120:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4215us (197778us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762945.883636:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1340 00000100:00100000:2.0:1789762945.883643:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469183181442 time 469183286605 state 0x2 ref 0 00000100:00100000:3.0:1789762945.888392:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762945.888606:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015141248 00000100:00100000:1.0:1789762945.888624:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015141248 00000100:00100000:1.0:1789762945.888757:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469183181442 time 469183286605 state 0x0 ref 1 00000100:00100000:1.0:1789762945.888781:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469383286605 time 469183286605 state 0x1 ref 1 00010000:00010000:3.0:1789762945.975340:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762946.079537:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469383286605 time 469383416359 state 0x0 ref 1 00000100:00100000:3.0:1789762946.079584:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015141248:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762946.079654:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008893c0a8/0x416193e012b05a6e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.079681:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762946.079688:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762946.079698:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008893c0a8/0x416193e012b05a6e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762946.079718:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008893c0a8/0x416193e012b05a6e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.084227:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000008893c0a8/0x416193e012b05a6e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.084244:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008893c0a8/0x416193e012b05a6e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.084252:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008893c0a8/0x416193e012b05a6e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.084258:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe78a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762946.084262:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008893c0a8/0x416193e012b05a6e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.084269:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008893c0a8/0x416193e012b05a6e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.084301:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008893c0a8/0x416193e012b05a6e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.084307:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008893c0a8/0x416193e012b05a6e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762946.084370:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015141248:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4804us (195990us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762946.084382:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1341 00000100:00100000:1.0:1789762946.084393:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469383286605 time 469383416359 state 0x2 ref 0 00000100:00100000:0.0:1789762946.087968:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762946.088018:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015141376 00000100:00100000:1.0:1789762946.088026:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015141376 00000100:00100000:1.0:1789762946.088075:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469383286605 time 469383416359 state 0x0 ref 1 00000100:00100000:1.0:1789762946.088109:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469583416359 time 469383416359 state 0x1 ref 1 00000100:00100000:3.0:1789762946.146594:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762946.146634:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762946.146647:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015141504 00000100:00100000:0.0:1789762946.146657:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015141504 00000100:00100000:1.0:1789762946.146724:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015141632 00000100:00100000:1.0:1789762946.146731:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015141632 00000100:00100000:3.0:1789762946.146733:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762946.146739:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 356 00000100:00100000:0.0:1789762946.146755:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015141504:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:3.0:1789762946.146777:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762946.146786:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015141760 00000100:00100000:1.0:1789762946.146790:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015141760 00000100:00100000:2.0:1789762946.146800:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 377 00000100:00100000:2.0:1789762946.146812:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702015141632:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762946.146826:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015141504:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 75us (242us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762946.146833:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 356 00000100:00100000:1.0:1789762946.146841:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 378 00000100:00100000:1.0:1789762946.146855:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015141760:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762946.146860:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702015141632:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 47us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762946.146889:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 377 00000100:00080000:0.0:1789762946.146895:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762946.146917:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015141760:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 65us (184us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762946.146926:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 378 00000100:00100000:2.0:1789762946.167228:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022196352:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762946.167269:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762946.167307:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022196352 00000100:00100000:0.0:1789762946.167313:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022196352 00000100:00100000:0.0:1789762946.167376:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:0.0:1789762946.167383:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022196352:12345-0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1789762946.167386:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00000024:0.0:1789762946.167417:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762946.167458:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022196352:12345-0@lo:13:osp-pre-0-0.0 Request processed in 76us (188us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762946.167463:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:2.0:1789762946.167499:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022196352:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762946.279600:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469583416359 time 469583496309 state 0x0 ref 1 00000100:00100000:0.0:1789762946.279625:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015141376:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762946.279662:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a75 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.279676:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762946.279680:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762946.279685:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a75 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.279700:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a75 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.282761:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a75 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.282772:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a75 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.282779:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a75 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.282789:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762946.282794:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a75 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.282800:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a75 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.282814:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a75 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.282819:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a75 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762946.282867:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015141376:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3252us (194899us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762946.282874:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1342 00000100:00100000:0.0:1789762946.282882:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469583416359 time 469583496309 state 0x2 ref 0 00000100:00100000:3.0:1789762946.284817:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762946.284849:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015141888 00000100:00100000:0.0:1789762946.284857:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015141888 00000100:00100000:0.0:1789762946.284890:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469583416359 time 469583496309 state 0x0 ref 1 00000100:00100000:0.0:1789762946.284901:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469783496309 time 469583496309 state 0x1 ref 1 00000100:00100000:2.0:1789762946.479687:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469783496309 time 469783583222 state 0x0 ref 1 00000100:00100000:2.0:1789762946.479709:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015141888:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762946.479745:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a7c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.479759:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762946.479763:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762946.479767:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a7c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.479779:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a7c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.482586:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a7c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.482595:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a7c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.482601:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a7c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.482605:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762946.482608:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a7c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.482631:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a7c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.482643:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a7c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762946.482648:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a7c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762946.482695:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015141888:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2993us (197879us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762946.482702:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1343 00000100:00100000:2.0:1789762946.482709:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469783496309 time 469783583222 state 0x2 ref 0 00000100:00100000:3.0:1789762946.484462:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762946.484512:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015142016 00000100:00100000:0.0:1789762946.484521:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015142016 00000100:00100000:0.0:1789762946.484556:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469783496309 time 469783583222 state 0x0 ref 1 00000100:00100000:0.0:1789762946.484575:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469983583222 time 469783583222 state 0x1 ref 1 00000100:00100000:0.0:1789762946.679287:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d8aa7aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022196480:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762946.679319:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86006300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022196608:0@lo:400:kworker.0 00000100:00100000:2.0:1789762946.679325:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86005f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022196864:0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.679345:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86005500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022196736:0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.679363:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762946.679381:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762946.679385:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762946.679418:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762946.679430:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86005880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022196992:0@lo:400:kworker.0 00000100:00100000:1.0:1789762946.679443:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e86006a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022197120:0@lo:400:kworker.0 00000100:00100000:2.0:1789762946.679455:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022196480 00000100:00100000:0.0:1789762946.679461:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762946.679464:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022196480 00000100:00100000:3.0:1789762946.679472:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022196864 00000100:00100000:1.0:1789762946.679478:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762946.679480:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022196864 00000100:00100000:0.0:1789762946.679518:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022196736 00000100:00100000:0.0:1789762946.679524:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022196736 00000100:00100000:1.0:1789762946.679528:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022197120 00000100:00100000:1.0:1789762946.679534:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022197120 00000100:00100000:3.0:1789762946.679535:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022196992 00000100:00100000:2.0:1789762946.679537:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 158 00000100:00100000:3.0:1789762946.679540:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022196992 00000100:00100000:2.0:1789762946.679548:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022196480:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762946.679563:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 379 00000100:00100000:1.0:1789762946.679579:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00002000:00000024:2.0:1789762946.679581:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762946.679584:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022196864:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.679590:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:1.0:1789762946.679592:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022197120:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.679607:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86006680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022196736:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762946.679632:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022196480:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (269us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762946.679639:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 158 00000100:00100000:2.0:1789762946.679679:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:2.0:1789762946.679687:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022196992:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.679699:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022196864:12345-0@lo:400:kworker.0 Request processed in 126us (316us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762946.679701:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022197120:12345-0@lo:400:kworker.0 Request processed in 113us (224us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762946.679703:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86006680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022196736:12345-0@lo:400:kworker.0 Request processed in 98us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762946.679719:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1789762946.679724:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 379 00000100:00100000:2.0:1789762946.679725:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022196992:12345-0@lo:400:kworker.0 Request processed in 35us (261us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762946.679732:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:2.0:1789762946.679739:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00100000:0.0:1789762946.679765:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022196608 00000100:00080000:2.0:1789762946.679769:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762946.679772:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022196608 00000100:00080000:2.0:1789762946.679775:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762946.679783:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86006a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022197120:0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.679786:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86005500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022196736:0@lo:400:kworker.0 00000100:00100000:2.0:1789762946.679799:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86005f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022196864:0@lo:400:kworker.0 00000100:00100000:0.0:1789762946.679820:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 357 00000100:00100000:0.0:1789762946.679831:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022196608:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762946.679840:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86005880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022196992:0@lo:400:kworker.0 00000100:00100000:1.0:1789762946.679842:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469983583222 time 469983736999 state 0x0 ref 1 00000100:00100000:3.0:1789762946.679862:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d8aa7aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022196480:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762946.679865:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e879c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015142016:12345-192.168.203.48@tcp:3:dd.500 00000100:00100000:0.0:1789762946.679882:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022196608:12345-0@lo:400:kworker.0 Request processed in 54us (504us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762946.679889:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 357 00010000:00010000:1.0:1789762946.679900:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a83 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762946.679911:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e86006300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022196608:0@lo:400:kworker.0 00010000:00010000:1.0:1789762946.679916:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.679922:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762946.679927:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a83 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.679939:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a83 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.682975:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a83 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.682985:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a83 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.682991:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a83 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.682996:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762946.683000:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a83 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.683006:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a83 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.683018:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a83 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762946.683023:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a83 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762946.683069:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e879c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015142016:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3211us (198607us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762946.683078:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1344 00000100:00100000:1.0:1789762946.683116:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469983583222 time 469983736999 state 0x2 ref 0 00000100:00100000:3.0:1789762946.685038:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762946.685075:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015142144 00000100:00100000:1.0:1789762946.685080:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015142144 00000100:00100000:1.0:1789762946.685123:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 469983583222 time 469983736999 state 0x0 ref 1 00000100:00100000:1.0:1789762946.685133:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470183736999 time 469983736999 state 0x1 ref 1 00000100:00100000:0.0:1789762946.879981:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470183736999 time 470183863259 state 0x0 ref 1 00000100:00100000:0.0:1789762946.880021:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015142144:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762946.880073:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a8a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.880125:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762946.880140:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762946.880148:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a8a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.880188:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a8a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.883491:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a8a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.883507:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a8a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.883514:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a8a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.883519:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762946.883523:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a8a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.883529:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a8a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.883548:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a8a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762946.883554:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a8a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762946.883612:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015142144:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3606us (198571us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762946.883620:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1345 00000100:00100000:0.0:1789762946.883631:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470183736999 time 470183863259 state 0x2 ref 0 00000100:00100000:3.0:1789762946.885697:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762946.885738:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015142272 00000100:00100000:0.0:1789762946.885747:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015142272 00000100:00100000:0.0:1789762946.885798:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470183736999 time 470183863259 state 0x0 ref 1 00000100:00100000:0.0:1789762946.885809:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470383863259 time 470183863259 state 0x1 ref 1 00010000:00010000:0.0:1789762946.999190:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762947.080113:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470383863259 time 470383986754 state 0x0 ref 1 00000100:00100000:2.0:1789762947.080162:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015142272:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:3.0:1789762947.080198:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789762947.080214:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a91 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.080232:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762947.080236:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762947.080242:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a91 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.080254:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a91 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.083697:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a91 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.083727:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a91 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.083739:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a91 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.083757:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762947.083764:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a91 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.083776:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a91 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.083799:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a91 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.083809:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05a91 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762947.083880:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015142272:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3747us (198183us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762947.083891:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1346 00000100:00100000:2.0:1789762947.083903:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470383863259 time 470383986754 state 0x2 ref 0 00000100:00100000:3.0:1789762947.086010:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762947.086063:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015142400 00000100:00100000:0.0:1789762947.086082:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015142400 00000100:00100000:0.0:1789762947.086146:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470383863259 time 470383986754 state 0x0 ref 1 00000100:00100000:0.0:1789762947.086159:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470583986754 time 470383986754 state 0x1 ref 1 00000100:00100000:1.0:1789762947.280291:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470583986754 time 470584171772 state 0x0 ref 1 00000100:00100000:1.0:1789762947.280326:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015142400:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762947.280385:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a98 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.280405:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762947.280410:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762947.280416:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a98 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.280428:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a98 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.284893:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a98 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.284904:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a98 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.284911:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a98 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.284916:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762947.284919:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a98 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.284925:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a98 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.284937:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a98 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.284942:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05a98 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762947.284997:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015142400:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4687us (198988us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762947.285003:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1347 00000100:00100000:1.0:1789762947.285012:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470583986754 time 470584171772 state 0x2 ref 0 00000100:00100000:3.0:1789762947.287061:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762947.287142:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015142528 00000100:00100000:1.0:1789762947.287151:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015142528 00000100:00100000:1.0:1789762947.287188:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470583986754 time 470584171772 state 0x0 ref 1 00000100:00100000:1.0:1789762947.287208:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470784171772 time 470584171772 state 0x1 ref 1 00000100:00100000:0.0:1789762947.480397:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470784171772 time 470784287782 state 0x0 ref 1 00000100:00100000:0.0:1789762947.480426:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015142528:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762947.480473:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a9f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.480491:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762947.480495:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762947.480501:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a9f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.480515:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a9f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.484345:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a9f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.484354:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a9f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.484361:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a9f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.484367:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762947.484372:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a9f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.484384:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a9f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.484401:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a9f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762947.484406:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05a9f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762947.484463:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86007100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015142528:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4049us (197409us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762947.484471:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1348 00000100:00100000:0.0:1789762947.484479:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470784171772 time 470784287782 state 0x2 ref 0 00000100:00100000:3.0:1789762947.486620:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762947.486658:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015142656 00000100:00100000:0.0:1789762947.486667:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015142656 00000100:00100000:0.0:1789762947.486710:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470784171772 time 470784287782 state 0x0 ref 1 00000100:00100000:0.0:1789762947.486723:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470984287782 time 470784287782 state 0x1 ref 1 00000100:00100000:2.0:1789762947.680495:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470984287782 time 470984384779 state 0x0 ref 1 00000100:00100000:2.0:1789762947.680538:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015142656:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762947.680589:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05aa6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.680607:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762947.680619:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762947.680626:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05aa6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.680641:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05aa6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.683645:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05aa6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.683662:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05aa6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.683669:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05aa6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.683674:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762947.683678:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05aa6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.683702:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05aa6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.683717:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05aa6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762947.683722:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05aa6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762947.683796:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015142656:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3262us (197172us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762947.683808:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1349 00000100:00100000:2.0:1789762947.683818:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470984287782 time 470984384779 state 0x2 ref 0 00000100:00100000:3.0:1789762947.685938:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762947.685998:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015142784 00000100:00100000:0.0:1789762947.686010:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015142784 00000100:00100000:0.0:1789762947.686066:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 470984287782 time 470984384779 state 0x0 ref 1 00000100:00100000:0.0:1789762947.686080:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471184384779 time 470984384779 state 0x1 ref 1 00000100:00100000:1.0:1789762947.880592:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471184384779 time 471184476964 state 0x0 ref 1 00000100:00100000:1.0:1789762947.880632:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015142784:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762947.880689:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05aad lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.880706:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762947.880711:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762947.880716:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05aad lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.880732:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05aad lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.884688:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05aad lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.884706:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05aad lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.884716:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05aad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.884724:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762947.884730:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05aad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.884742:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05aad lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.884767:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05aad lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762947.884775:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05aad lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762947.884841:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86004e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015142784:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4224us (198906us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762947.884851:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1350 00000100:00100000:1.0:1789762947.884862:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471184384779 time 471184476964 state 0x2 ref 0 00000100:00100000:3.0:1789762947.887058:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762947.887124:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015142912 00000100:00100000:1.0:1789762947.887134:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015142912 00000100:00100000:1.0:1789762947.887186:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471184384779 time 471184476964 state 0x0 ref 1 00000100:00100000:1.0:1789762947.887202:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471384476964 time 471184476964 state 0x1 ref 1 00010000:00010000:0.0:1789762948.023314:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762948.080721:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471384476964 time 471384605916 state 0x0 ref 1 00000100:00100000:0.0:1789762948.080762:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86006300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015142912:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762948.080828:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05ab4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.080850:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762948.080854:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762948.080859:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05ab4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.080874:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05ab4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.084292:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05ab4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.084309:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05ab4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.084316:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05ab4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.084321:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80030a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762948.084326:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05ab4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.084332:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05ab4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.084347:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05ab4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.084352:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000027c6d3a7/0x416193e012b05ab4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762948.084410:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86006300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015142912:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3665us (197353us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762948.084419:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1351 00000100:00100000:0.0:1789762948.084430:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471384476964 time 471384605916 state 0x2 ref 0 00000100:00100000:3.0:1789762948.086397:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762948.086435:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015143040 00000100:00100000:0.0:1789762948.086443:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015143040 00000100:00100000:0.0:1789762948.086482:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471384476964 time 471384605916 state 0x0 ref 1 00000100:00100000:0.0:1789762948.086493:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471584605916 time 471384605916 state 0x1 ref 1 00000100:00100000:2.0:1789762948.280823:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471584605916 time 471584704621 state 0x0 ref 1 00000100:00100000:2.0:1789762948.280858:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015143040:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762948.280908:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05abb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.280927:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762948.280931:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762948.280938:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05abb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.280973:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05abb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.284337:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05abb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.284352:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05abb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.284362:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05abb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.284369:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762948.284374:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05abb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.284385:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05abb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.284405:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05abb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.284415:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05abb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762948.284490:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86005f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015143040:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3648us (198095us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762948.284498:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1352 00000100:00100000:2.0:1789762948.284507:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471584605916 time 471584704621 state 0x2 ref 0 00000100:00100000:1.0:1789762948.286675:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762948.286755:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015143168 00000100:00100000:0.0:1789762948.286765:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015143168 00000100:00100000:0.0:1789762948.286809:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471584605916 time 471584704621 state 0x0 ref 1 00000100:00100000:0.0:1789762948.286825:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471784704621 time 471584704621 state 0x1 ref 1 00000100:00100000:1.0:1789762948.480931:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471784704621 time 471784814991 state 0x0 ref 1 00000100:00100000:1.0:1789762948.480964:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015143168:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762948.481022:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05ac2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.481045:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762948.481050:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762948.481058:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05ac2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762948.481091:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05ac2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.484959:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05ac2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.484976:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05ac2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.484986:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05ac2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.484993:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfa4d600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762948.484998:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05ac2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.485007:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05ac2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.485028:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05ac2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.485035:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ba83da8/0x416193e012b05ac2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762948.485178:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015143168:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4224us (198501us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762948.485189:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1353 00000100:00100000:2.0:1789762948.485200:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471784704621 time 471784814991 state 0x2 ref 0 00000100:00100000:3.0:1789762948.487456:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762948.487514:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015143296 00000100:00100000:1.0:1789762948.487526:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015143296 00000100:00100000:1.0:1789762948.487577:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471784704621 time 471784814991 state 0x0 ref 1 00000100:00100000:1.0:1789762948.487589:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471984814991 time 471784814991 state 0x1 ref 1 00000100:00100000:0.0:1789762948.681072:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471984814991 time 471984956705 state 0x0 ref 1 00000100:00100000:0.0:1789762948.681151:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86006a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015143296:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762948.681222:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b05ac9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.681245:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762948.681251:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762948.681277:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b05ac9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762948.681299:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b05ac9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.685795:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b05ac9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.685813:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b05ac9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.685824:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b05ac9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.685834:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e80031a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762948.685843:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b05ac9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.685857:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b05ac9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.685888:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b05ac9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.685903:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000df0d417/0x416193e012b05ac9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762948.686007:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86006a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015143296:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4877us (198552us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762948.686020:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1354 00000100:00100000:2.0:1789762948.686033:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471984814991 time 471984956705 state 0x2 ref 0 00000100:00100000:3.0:1789762948.688170:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762948.688337:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015143424 00000100:00100000:0.0:1789762948.688350:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015143424 00000100:00100000:0.0:1789762948.688409:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 471984814991 time 471984956705 state 0x0 ref 1 00000100:00100000:0.0:1789762948.688425:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 472184956705 time 471984956705 state 0x1 ref 1 00000100:00100000:2.0:1789762948.881216:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 472184956705 time 472185101385 state 0x0 ref 1 00000100:00100000:2.0:1789762948.881277:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015143424:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762948.881339:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.881361:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762948.881366:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762948.881373:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.881394:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.884872:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.884890:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.884914:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.884920:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762948.884924:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.884930:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.884947:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762948.884953:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762948.885016:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015143424:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3775us (196844us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762948.885042:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1355 00000100:00100000:2.0:1789762948.885056:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 472184956705 time 472185101385 state 0x2 ref 0 00000100:00100000:1.0:1789762948.887194:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762948.887242:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015143552 00000100:00100000:0.0:1789762948.887251:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015143552 00000100:00100000:0.0:1789762948.887315:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 472184956705 time 472185101385 state 0x0 ref 1 00000100:00100000:0.0:1789762948.887328:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 472385101385 time 472185101385 state 0x1 ref 1 00010000:00010000:1.0:1789762949.047199:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762949.081324:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 472385101385 time 472385220662 state 0x0 ref 1 00000100:00100000:2.0:1789762949.081373:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015143552:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762949.081434:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.081449:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762949.081453:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762949.081459:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.081471:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789762949.081481:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789762949.086031:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.086048:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.086056:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.086062:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762949.086066:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.086073:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.086141:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.086150:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000a4e9431/0x416193e012b05ad7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762949.086222:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015143552:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4880us (199026us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762949.086242:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1356 00000100:00100000:2.0:1789762949.086252:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 472385101385 time 472385220662 state 0x2 ref 0 00000100:00100000:0.0:1789762949.110162:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762949.110251:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015143680 00000100:00100000:2.0:1789762949.110274:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015143680 00000100:00100000:2.0:1789762949.110331:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 34 00000100:00100000:2.0:1789762949.110340:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:5593:x1876702015143680:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762949.110352:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762949.110385:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000140542ee/0x416193e012b0581b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd85b expref: 6 pid: 6881 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.110410:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762949.110457:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:5593:x1876702015143680:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 118us (302us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762949.110463:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 34 00000100:00100000:3.0:1789762949.118472:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762949.118539:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015143808 00000100:00100000:1.0:1789762949.118547:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015143808 00000100:00100000:1.0:1789762949.118594:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 360 00000100:00100000:1.0:1789762949.118603:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:12244:x1876702015143808:12345-192.168.203.48@tcp:101:bash.0 00000100:00080000:3.0:1789762949.118608:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789762949.118617:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762949.118664:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05ade lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd862 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.118734:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05ae5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.118749:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05ae5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.118761:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05ae5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762949.118791:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00000004:00200000:1.0:1789762949.118794:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762949.118808:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05ae5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.118826:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05ade lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd862 expref: 11 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.118838:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05ae5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd862 expref: 10 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.118844:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981ea0aa3e00) 00010000:00010000:1.0:1789762949.118846:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762949.118849:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc600, rc 0) 00000100:00100000:1.0:1789762949.118881:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:12244:x1876702015143808:12345-192.168.203.48@tcp:101:bash.0 Request processed in 281us (418us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762949.118887:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 360 00000100:00100000:1.0:1789762949.122473:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762949.122529:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015144064 00000100:00100000:3.0:1789762949.122540:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015144064 00000100:00100000:3.0:1789762949.122609:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 361 00000100:00100000:3.0:1789762949.122620:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15468:x1876702015144064:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:3.0:1789762949.122638:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762949.122654:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05aec lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd869 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762949.122771:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00010000:00010000:3.0:1789762949.123012:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05aec lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd869 expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.123024:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:3.0:1789762949.123026:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762949.123030:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05aec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd869 expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.123040:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aaa00, rc 0) 00000100:00100000:3.0:1789762949.123109:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15468:x1876702015144064:12345-192.168.203.48@tcp:101:rm.0 Request processed in 491us (637us total) trans 4294967609 rc 301/301 00000100:00100000:3.0:1789762949.123115:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 361 00000100:00100000:0.0:1789762949.124841:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762949.124878:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015144192 00000100:00100000:1.0:1789762949.124885:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015144192 00000100:00100000:1.0:1789762949.124916:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 115 00000100:00100000:1.0:1789762949.124922:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15468:x1876702015144192:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:1.0:1789762949.127128:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15468:x1876702015144192:12345-192.168.203.48@tcp:37:rm.0 Request processed in 2205us (2288us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762949.127134:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 115 00000100:00100000:3.0:1789762949.128616:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762949.128658:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015144448 00000100:00100000:0.0:1789762949.128676:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015144448 00000100:00100000:0.0:1789762949.128705:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 116 00000100:00100000:0.0:1789762949.128711:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15468:x1876702015144448:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:0.0:1789762949.128883:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15468:x1876702015144448:12345-192.168.203.48@tcp:35:rm.0 Request processed in 167us (265us total) trans 4294967610 rc 0/0 00000100:00100000:0.0:1789762949.128891:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 116 00000100:00100000:0.0:1789762949.129845:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762949.129901:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015144704 00000100:00100000:3.0:1789762949.129910:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015144704 00000100:00100000:3.0:1789762949.129951:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 362 00000100:00100000:3.0:1789762949.129965:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15468:x1876702015144704:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:3.0:1789762949.129981:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762949.129992:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05afa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd870 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.130058:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05b01 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.130073:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05b01 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.130109:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05b01 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762949.130122:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00010000:00010000:3.0:1789762949.130220:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05afa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd870 expref: 15 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.130242:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05b01 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd870 expref: 14 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.130272:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981ea0aa3e00) 00010000:00010000:3.0:1789762949.130276:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1789762949.130281:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880ab800, rc 0) 00000100:00100000:3.0:1789762949.130342:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15468:x1876702015144704:12345-192.168.203.48@tcp:101:rm.0 Request processed in 382us (495us total) trans 4294967611 rc 0/0 00000100:00100000:3.0:1789762949.130351:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 362 00000100:00100000:2.0:1789762949.131205:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762949.131298:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015144832 00000100:00100000:1.0:1789762949.131304:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015144832 00000100:00100000:1.0:1789762949.131334:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 363 00000100:00100000:1.0:1789762949.131340:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8aa7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15468:x1876702015144832:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:1.0:1789762949.131373:0:6611:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7610564608 00000100:00100000:1.0:1789762949.131407:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8aa7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15468:x1876702015144832:12345-192.168.203.48@tcp:41:rm.0 Request processed in 67us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762949.131412:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 363 00000100:00100000:1.0:1789762949.132150:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762949.132190:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015145088 00000100:00100000:0.0:1789762949.132194:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015145088 00000100:00100000:0.0:1789762949.132221:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 117 00000100:00100000:0.0:1789762949.132230:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15468:x1876702015145088:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:0.0:1789762949.132424:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15468:x1876702015145088:12345-192.168.203.48@tcp:35:rm.0 Request processed in 195us (274us total) trans 4294967612 rc 0/0 00000100:00100000:0.0:1789762949.132430:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 117 00000100:00100000:0.0:1789762949.133033:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762949.133066:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015145344 00000100:00100000:3.0:1789762949.133072:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015145344 00000100:00100000:3.0:1789762949.133116:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 364 00000100:00100000:3.0:1789762949.133123:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15468:x1876702015145344:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:3.0:1789762949.133158:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1789762949.133325:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a6000/0x416193e012b0580d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd854 expref: 15 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.133344:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981d865a7000/0x416193e012b05520 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd84d expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.133352:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762949.133362:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc600/0x416193e012b05ae5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd862 expref: 13 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.133368:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981ea0aa3e00) 00010000:00010000:3.0:1789762949.133370:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762949.133383:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05b0f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.133390:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05b0f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.133396:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05b0f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.133418:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05b16 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.133424:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05b16 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.133429:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05b16 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.133659:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05b1d lrc: 3/0,1 mode: --/EX res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.133666:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05b1d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.133671:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05b1d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762949.134545:0:5806:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].24, 64 off9728 00000004:00080000:3.0:1789762949.134554:0:5806:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 472438466621ns, next at 473438464841ns, rc = 0 00010000:00010000:3.0:1789762949.134599:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05b24 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134606:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05b24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134611:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05b24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134616:0:5806:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05b24 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134621:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05b24 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134625:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05b24 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134642:0:5806:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762949.134653:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05b1d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134657:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05b1d lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x3c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134663:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05b1d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134668:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762949.134671:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762949.134672:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05b1d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762949.134679:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05b24 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134680:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05b1d lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x3c:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134699:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff981e9272c600) 00010000:00010000:3.0:1789762949.134703:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789762949.134707:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aba00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762949.134709:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05b1d lrc: 2/0,0 mode: --/EX res: [0x200000401:0x3c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762949.134713:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05b24 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134733:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05b1d lrc: 0/0,0 mode: --/EX res: [0x200000401:0x3c:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762949.134740:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05b24 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134753:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05b0f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762949.134756:0:6663:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff981e9272c600) 00010000:00010000:0.0:1789762949.134764:0:6663:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789762949.134767:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05b0f lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762949.134771:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05b24 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x3c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134773:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05b0f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134787:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762949.134790:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05b24 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134792:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05b0f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134811:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05b0f lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762949.134816:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762949.134819:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981ea0aa3e00) 00010000:00010000:0.0:1789762949.134820:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762949.134823:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762949.134824:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762949.134829:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05b0f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.134835:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05b0f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762949.134848:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d9c0dfe7 reply state 000000005deac18a transno 4294967613 ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05b16 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762949.134856:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762949.134909:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15468:x1876702015145344:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1782us (1871us total) trans 4294967613 rc 0/0 00000100:00100000:3.0:1789762949.134915:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 364 00010000:00010000:1.0:1789762949.135502:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05b16 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.135511:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05b16 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.135516:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05b16 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.135521:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aac00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762949.135523:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05b16 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.135528:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05b16 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.135535:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x4009f790 (ffff981e9272ce00) 00010000:00010000:1.0:1789762949.135537:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762949.135539:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05b16 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.135544:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05b16 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762949.135967:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762949.135997:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015145600 00000100:00100000:3.0:1789762949.136003:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015145600 00000100:00100000:3.0:1789762949.136076:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 118 00000100:00100000:3.0:1789762949.136114:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7ad7c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:15468:x1876702015145600:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762949.136219:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7ad7c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15468:x1876702015145600:12345-192.168.203.48@tcp:35:rm.0 Request processed in 134us (251us total) trans 4294967614 rc 0/0 00000100:00100000:3.0:1789762949.136229:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 118 00000100:00100000:2.0:1789762949.136853:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762949.136885:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015145856 00000100:00100000:3.0:1789762949.136889:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015145856 00000100:00100000:3.0:1789762949.136913:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 365 00000100:00100000:3.0:1789762949.136919:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad83b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15468:x1876702015145856:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:3.0:1789762949.136945:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1789762949.136955:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05b01 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd870 expref: 14 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.136977:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05504 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd846 expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.136985:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981ea0aa3e00) 00010000:00010000:3.0:1789762949.136993:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfa4c800/0x416193e012b054a2 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd831 expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.136999:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179e00) 00010000:00010000:3.0:1789762949.137005:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762949.137018:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b2b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137027:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b2b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137035:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b2b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137068:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b32 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137075:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b32 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137080:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b32 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137310:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05b39 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137320:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05b39 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137325:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05b39 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762949.137659:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00010000:00010000:3.0:1789762949.137667:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05b39 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137672:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05b39 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x3a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137676:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05b39 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137679:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aa400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762949.137682:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05b39 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137685:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05b39 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x3a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137692:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff981e9272c100) 00010000:00010000:3.0:1789762949.137694:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789762949.137696:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05b39 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x3a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137707:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05b39 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x3a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137723:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b2b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137730:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b2b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137738:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b2b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137744:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aaa00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762949.137747:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b2b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137754:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b2b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137761:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:3.0:1789762949.137762:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762949.137765:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b2b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.137769:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b2b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762949.137776:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d533d0d5 reply state 000000004d0146d9 transno 4294967615 ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b32 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762949.137781:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762949.137814:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad83b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:15468:x1876702015145856:12345-192.168.203.48@tcp:36:rm.0 Request processed in 895us (962us total) trans 4294967615 rc 0/0 00000100:00100000:3.0:1789762949.137820:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 365 00010000:00010000:3.0:1789762949.138280:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b32 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.138286:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b32 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.138291:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b32 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.138294:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aa600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762949.138296:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b32 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.138300:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b32 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.138307:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272cb00) 00010000:00010000:3.0:1789762949.138309:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762949.138311:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b32 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762949.138315:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b32 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762949.248628:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967612 is committed 00000001:00080000:2.0:1789762949.248643:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967615 is committed 00000004:00080000:2.0:1789762949.248650:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 31 committed 00000100:00100000:2.0:1789762949.248759:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789762949.248777:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4ab5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022197248:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789762949.248842:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762949.248868:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022197248 00000100:00100000:1.0:1789762949.248875:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022197248 00000100:00080000:3.0:1789762949.248916:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762949.248932:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789762949.248943:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb494e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022197248:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1789762949.248964:0:6647:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:24 count 1 00002000:00010000:1.0:1789762949.248991:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 104857600, mtime: 0x6aad9d71, atime: 0x0, ctime: 0x6aad9d71, blocks: 0x3200b 00010000:00010000:1.0:1789762949.248998:0:6647:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05b40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.249012:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=472s v=(0 0) 00010000:00010000:1.0:1789762949.249017:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762949.249023:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05b40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.249038:0:6647:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05b40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.249046:0:6647:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05b40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.249052:0:6647:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05b40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762949.249069:0:6647:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789762949.249080:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762949.249114:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05b40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.249123:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762949.249128:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05b40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.249136:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05b40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.249153:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05b40 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.249159:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c253e283/0x416193e012b05b40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762949.249171:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762949.249172:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762949.249173:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762949.249615:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb494e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022197248:12345-0@lo:6:osp-syn-0-0.0 Request processed in 674us (774us total) trans 4294968251 rc 0/0 00000100:00100000:1.0:1789762949.249623:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000004:00080000:2.0:1789762949.249646:0:3274:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981eb4ab5c00/2, rc 0, transno 4294968251 00000100:00100000:2.0:1789762949.249660:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4ab5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022197248:0@lo:6:osp-syn-0-0.0 00000004:00080000:1.0:1789762949.971842:0:17165:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 473275752297ns, next at 473438464841ns, rc = 0 00000004:00080000:1.0:1789762949.971928:0:17165:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 473275840210ns, next at 435143167655ns, rc = 0 00000004:00080000:1.0:1789762949.972997:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 32 committed 00000004:00080000:3.0:1789762949.973671:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 25 committed 00010000:00010000:0.0:1789762950.071187:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789762950.281191:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968251 is committed 00010000:00010000:1.0:1789762951.095273:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762951.266259:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762951.266301:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762951.266325:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762951.266347:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015146112 00000100:00100000:0.0:1789762951.266356:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015145984 00000100:00100000:2.0:1789762951.266357:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762951.266359:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015146112 00000100:00100000:0.0:1789762951.266363:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015145984 00000100:00100000:2.0:1789762951.266388:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015146240 00000100:00100000:2.0:1789762951.266394:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015146240 00000100:00100000:3.0:1789762951.266395:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015146368 00000100:00100000:3.0:1789762951.266401:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015146368 00000100:00100000:0.0:1789762951.266415:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 358 00000100:00100000:0.0:1789762951.266428:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015145984:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762951.266435:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 382 00000100:00100000:2.0:1789762951.266436:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 187 00000100:00100000:3.0:1789762951.266451:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 383 00000100:00100000:2.0:1789762951.266458:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015146240:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762951.266459:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015146112:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762951.266467:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad82a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015146368:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762951.266493:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2364:x1876702015145984:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 69us (189us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762951.266500:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 358 00000100:00100000:2.0:1789762951.266505:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015146240:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 57us (146us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762951.266510:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad82a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2363:x1876702015146368:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 48us (181us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762951.266519:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 187 00000100:00100000:3.0:1789762951.266521:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 383 00000100:00100000:1.0:1789762951.266523:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2363:x1876702015146112:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 70us (277us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762951.266537:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 382 00000100:00080000:1.0:1789762951.266566:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789762951.266572:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762951.287232:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb18700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022197376:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762951.287292:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762951.287335:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022197376 00000100:00100000:0.0:1789762951.287341:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022197376 00000100:00080000:1.0:1789762951.287385:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762951.287390:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00000100:00100000:0.0:1789762951.287398:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022197376:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762951.287424:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00100000:0.0:1789762951.287463:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022197376:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (172us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762951.287468:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00000100:00100000:2.0:1789762951.287497:0:3274:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294968251 gen 2 00000004:00080000:2.0:1789762951.287504:0:3274:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981eb4ab5c00, transno 4294968251 00000100:00100000:2.0:1789762951.287525:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb18700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022197376:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789762951.799273:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb19f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022197632:0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.799273:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022197760:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.799294:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4ab5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022197504:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762951.799295:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb18e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022197888:0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.799355:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762951.799356:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762951.799362:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762951.799387:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022197632 00000100:00100000:1.0:1789762951.799393:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022197632 00000100:00100000:0.0:1789762951.799425:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022198016:0@lo:400:kworker.0 00000100:00100000:3.0:1789762951.799433:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022198272:0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.799434:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb18700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022198144:0@lo:400:kworker.0 00000100:00100000:1.0:1789762951.799439:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:1.0:1789762951.799451:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022197632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.799457:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762951.799475:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762951.799497:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022197760 00000100:00100000:3.0:1789762951.799502:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022197760 00000100:00100000:1.0:1789762951.799517:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022197632:12345-0@lo:400:kworker.0 Request processed in 68us (161us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762951.799523:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022197504 00000100:00100000:1.0:1789762951.799524:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00100000:2.0:1789762951.799527:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb19f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022197632:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.799530:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022197504 00000100:00100000:3.0:1789762951.799540:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022198144 00000100:00100000:3.0:1789762951.799544:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022198144 00000100:00100000:2.0:1789762951.799579:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762951.799584:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:3.0:1789762951.799591:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1789762951.799596:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022197760:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.799605:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:3.0:1789762951.799606:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad83100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022198144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.799619:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022197888 00000100:00100000:0.0:1789762951.799622:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022197504:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762951.799628:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022197760:12345-0@lo:400:kworker.0 Request processed in 37us (275us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762951.799636:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:2.0:1789762951.799638:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022197888 00002000:00000024:0.0:1789762951.799654:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762951.799665:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad83100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022198144:12345-0@lo:400:kworker.0 Request processed in 63us (190us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762951.799675:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1789762951.799678:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022198016 00000100:00100000:2.0:1789762951.799681:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:1.0:1789762951.799683:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022198016 00000100:00100000:2.0:1789762951.799695:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022197888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.799715:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022197504:12345-0@lo:13:osp-pre-1-0.0 Request processed in 98us (355us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762951.799721:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:3.0:1789762951.799725:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb18700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022198144:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.799726:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:1.0:1789762951.799733:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022198016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762951.799747:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022197888:12345-0@lo:400:kworker.0 Request processed in 55us (167us total) trans 0 rc 0/0 00000100:00080000:0.0:1789762951.799755:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762951.799756:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00080000:0.0:1789762951.799762:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789762951.799773:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022198016:12345-0@lo:400:kworker.0 Request processed in 42us (315us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762951.799780:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:3.0:1789762951.799782:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb18e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022197888:0@lo:400:kworker.0 00000100:00100000:0.0:1789762951.799804:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4ab5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022197504:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762951.799813:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762951.799829:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022198016:0@lo:400:kworker.0 00000100:00100000:1.0:1789762951.799834:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022198272 00000100:00100000:1.0:1789762951.799839:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022198272 00000100:00100000:2.0:1789762951.799850:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022197760:0@lo:400:kworker.0 00000100:00100000:1.0:1789762951.799868:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:1.0:1789762951.799875:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022198272:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762951.799903:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022198272:12345-0@lo:400:kworker.0 Request processed in 30us (95us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762951.799908:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:2.0:1789762951.799926:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022198272:0@lo:400:kworker.0 00010000:00010000:0.0:1789762952.119182:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762953.143253:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762954.167194:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762955.191371:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762956.215209:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762956.386717:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762956.386749:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762956.386771:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762956.386792:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015146496 00000100:00100000:3.0:1789762956.386807:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015146624 00000100:00100000:0.0:1789762956.386813:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762956.386815:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015146496 00000100:00100000:3.0:1789762956.386815:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015146624 00000100:00100000:1.0:1789762956.386838:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015146752 00000100:00100000:1.0:1789762956.386845:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015146752 00000100:00100000:0.0:1789762956.386861:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015146880 00000100:00100000:0.0:1789762956.386867:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015146880 00000100:00100000:2.0:1789762956.386873:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 360 00000100:00100000:3.0:1789762956.386880:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 387 00000100:00100000:2.0:1789762956.386889:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015146496:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762956.386893:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 190 00000100:00100000:3.0:1789762956.386896:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015146624:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762956.386911:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 388 00000100:00100000:1.0:1789762956.386913:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015146752:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762956.386923:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015146880:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762956.386965:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015146624:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 75us (212us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762956.386967:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015146496:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 83us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762956.386969:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015146752:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 64us (196us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762956.386977:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015146880:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 54us (160us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762956.386990:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 360 00000100:00100000:1.0:1789762956.386992:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 190 00000100:00100000:3.0:1789762956.386998:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 387 00000100:00100000:0.0:1789762956.387005:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 388 00000100:00080000:3.0:1789762956.387025:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762956.387030:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789762956.407260:0:3273:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789762956.407279:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad80700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022198400:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762956.407334:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762956.407392:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022198400 00000100:00100000:1.0:1789762956.407397:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022198400 00000100:00100000:1.0:1789762956.407438:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:1.0:1789762956.407452:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022198400:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789762956.407477:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00080000:3.0:1789762956.407483:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762956.407518:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022198400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (185us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762956.407524:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00100000:0.0:1789762956.407565:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad80700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022198400:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762956.919270:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eb4ab4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022198528:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762956.919287:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022198656:0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.919332:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022198784:0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.919342:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762956.919343:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762956.919399:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762956.919405:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022198656 00000100:00100000:2.0:1789762956.919414:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022198656 00000100:00100000:3.0:1789762956.919420:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb18700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022199168:0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.919422:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb18e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022199040:0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.919430:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb19f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022199296:0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.919458:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762956.919467:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 361 00000100:00100000:0.0:1789762956.919474:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762956.919482:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022198784 00000100:00100000:2.0:1789762956.919485:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022198656:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.919487:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022198784 00000100:00100000:3.0:1789762956.919498:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022198528 00000100:00100000:3.0:1789762956.919502:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022198528 00000100:00100000:1.0:1789762956.919525:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022199168 00000100:00100000:1.0:1789762956.919531:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022199168 00000100:00100000:2.0:1789762956.919549:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022198656:12345-0@lo:400:kworker.0 Request processed in 68us (210us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762956.919559:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 361 00000100:00100000:3.0:1789762956.919564:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00100000:3.0:1789762956.919578:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022198528:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762956.919579:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:1.0:1789762956.919593:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022198784:12345-0@lo:400:kworker.0 00002000:00000024:3.0:1789762956.919609:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:2.0:1789762956.919615:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:2.0:1789762956.919625:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022199168:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.919642:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022198784:12345-0@lo:400:kworker.0 Request processed in 51us (243us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762956.919645:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022199040 00000100:00100000:1.0:1789762956.919651:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00100000:0.0:1789762956.919652:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022199040 00000100:00100000:3.0:1789762956.919657:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022198528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762956.919664:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00100000:2.0:1789762956.919688:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3272:x1876702022199168:12345-0@lo:400:kworker.0 Request processed in 64us (229us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762956.919694:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789762956.919699:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00100000:1.0:1789762956.919701:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1789762956.919701:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762956.919707:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:3.0:1789762956.919715:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022198912:0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.919722:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022199040:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.919743:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022198784:0@lo:400:kworker.0 00000100:00100000:2.0:1789762956.919743:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022198656:0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.919751:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762956.919778:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb18e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022199040:0@lo:400:kworker.0 00000100:00100000:1.0:1789762956.919780:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022199296 00000100:00100000:0.0:1789762956.919781:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022199040:12345-0@lo:400:kworker.0 Request processed in 63us (310us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762956.919785:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022199296 00000100:00100000:0.0:1789762956.919794:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:3.0:1789762956.919799:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022198912 00000100:00100000:3.0:1789762956.919803:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022198912 00000100:00100000:0.0:1789762956.919815:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:0.0:1789762956.919826:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022199296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.919843:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:2.0:1789762956.919847:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb18700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3272:1876702022199168:0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.919854:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022198912:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762956.919869:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86d0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022199296:12345-0@lo:400:kworker.0 Request processed in 46us (169us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762956.919874:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eb4ab4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022198528:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789762956.919878:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:1.0:1789762956.919882:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb19f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022199296:0@lo:400:kworker.0 00000100:00100000:3.0:1789762956.919902:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022198912:12345-0@lo:400:kworker.0 Request processed in 52us (154us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762956.919911:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00080000:2.0:1789762956.919922:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762956.919940:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022198912:0@lo:400:kworker.0 00010000:00010000:2.0:1789762957.239266:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762958.263257:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762959.287199:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762960.311211:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762961.335212:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762961.506218:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762961.506247:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762961.506270:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762961.506280:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015147008 00000100:00100000:1.0:1789762961.506290:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015147008 00000100:00100000:3.0:1789762961.506300:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762961.506305:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015147136 00000100:00100000:0.0:1789762961.506312:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015147136 00000100:00100000:2.0:1789762961.506322:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015147392 00000100:00100000:2.0:1789762961.506330:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015147392 00000100:00100000:3.0:1789762961.506348:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015147264 00000100:00100000:1.0:1789762961.506349:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 362 00000100:00100000:3.0:1789762961.506355:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015147264 00000100:00100000:1.0:1789762961.506362:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015147008:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762961.506372:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 392 00000100:00100000:2.0:1789762961.506387:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 393 00000100:00100000:0.0:1789762961.506388:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015147136:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762961.506393:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 193 00000100:00100000:2.0:1789762961.506402:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015147392:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762961.506408:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4ab5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015147264:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762961.506427:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015147008:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 67us (216us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762961.506434:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015147136:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 53us (183us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762961.506438:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 362 00000100:00100000:0.0:1789762961.506448:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 392 00000100:00100000:2.0:1789762961.506450:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015147392:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 50us (176us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762961.506458:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4ab5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015147264:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 54us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762961.506460:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 393 00000100:00100000:3.0:1789762961.506472:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 193 00000100:00080000:0.0:1789762961.506473:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789762961.506479:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762961.527255:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022199424:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762961.527294:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762961.527329:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022199424 00000100:00100000:1.0:1789762961.527335:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022199424 00000100:00080000:0.0:1789762961.527355:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762961.527377:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:1.0:1789762961.527384:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022199424:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789762961.527407:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3859808256 avail 3857711104 00000100:00100000:1.0:1789762961.527445:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb16300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022199424:12345-0@lo:13:osp-pre-0-0.0 Request processed in 63us (151us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762961.527451:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:3.0:1789762961.527483:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad83480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022199424:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789762962.039223:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022199552:0@lo:400:kworker.0 00000100:00100000:3.0:1789762962.039232:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb19f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022199680:0@lo:400:kworker.0 00000100:00100000:1.0:1789762962.039277:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb18700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022199808:0@lo:400:kworker.0 00000100:00100000:0.0:1789762962.039295:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762962.039302:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762962.039327:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb18e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022199936:0@lo:400:kworker.0 00000100:00100000:1.0:1789762962.039360:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762962.039373:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022200064:0@lo:400:kworker.0 00000100:00100000:3.0:1789762962.039375:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981eadb1a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022200192:0@lo:400:kworker.0 00000100:00100000:1.0:1789762962.039404:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022199936 00000100:00100000:1.0:1789762962.039411:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022199936 00000100:00100000:0.0:1789762962.039425:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762962.039429:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762962.039459:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022200192 00000100:00100000:1.0:1789762962.039463:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022200192 00000100:00100000:0.0:1789762962.039489:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022199552 00000100:00100000:0.0:1789762962.039495:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022199552 00000100:00100000:3.0:1789762962.039497:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022199680 00000100:00100000:1.0:1789762962.039504:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:3.0:1789762962.039504:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022199680 00000100:00100000:1.0:1789762962.039517:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe70e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022199936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762962.039534:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 363 00000100:00100000:0.0:1789762962.039543:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e850b1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022199552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762962.039554:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022200064 00000100:00100000:3.0:1789762962.039559:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022200064 00000100:00100000:1.0:1789762962.039573:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe70e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022199936:12345-0@lo:400:kworker.0 Request processed in 59us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762962.039580:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:0.0:1789762962.039607:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e850b1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3271:x1876702022199552:12345-0@lo:400:kworker.0 Request processed in 63us (310us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762962.039613:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:3.0:1789762962.039615:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:0.0:1789762962.039621:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 363 00000100:00100000:1.0:1789762962.039627:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022200192:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762962.039633:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f71c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022199680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762962.039660:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:0.0:1789762962.039670:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022200064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762962.039688:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022200192:12345-0@lo:400:kworker.0 Request processed in 63us (263us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762962.039695:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00100000:3.0:1789762962.039708:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f71c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022199680:12345-0@lo:400:kworker.0 Request processed in 79us (406us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762962.039718:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:0.0:1789762962.039722:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022200064:12345-0@lo:400:kworker.0 Request processed in 52us (300us total) trans 0 rc 0/0 00000100:00080000:1.0:1789762962.039723:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789762962.039731:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762962.039732:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:1.0:1789762962.039761:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762962.039782:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022200064:0@lo:400:kworker.0 00000100:00100000:3.0:1789762962.039802:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022200192:0@lo:400:kworker.0 00000100:00100000:0.0:1789762962.039804:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb1ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3271:1876702022199552:0@lo:400:kworker.0 00000100:00100000:1.0:1789762962.039818:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022199808 00000100:00100000:1.0:1789762962.039825:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022199808 00000100:00100000:3.0:1789762962.039834:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb19f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022199680:0@lo:400:kworker.0 00000100:00100000:0.0:1789762962.039875:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb18e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022199936:0@lo:400:kworker.0 00000100:00100000:1.0:1789762962.039876:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:1.0:1789762962.039891:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfe71f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022199808:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762962.039918:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762962.039939:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfe71f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022199808:12345-0@lo:400:kworker.0 Request processed in 54us (180us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762962.039946:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:3.0:1789762962.039968:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981eadb18700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022199808:0@lo:400:kworker.0 00000100:00100000:1.0:1789762962.040182:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d846f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022200320:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762962.040210:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762962.040237:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022200320 00000100:00100000:3.0:1789762962.040242:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022200320 00000100:00100000:3.0:1789762962.040277:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:3.0:1789762962.040286:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022200320:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789762962.040308:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762962.040354:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022200320:12345-0@lo:13:osp-pre-1-0.0 Request processed in 71us (142us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762962.040361:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00100000:1.0:1789762962.040387:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d846f2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022200320:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789762962.359200:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762963.028449:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:3.0:1789762963.028458:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:3.0:1789762963.028465:0:17485:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762963.028472:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:3.0:1789762963.028475:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:3.0:1789762963.028478:0:17485:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981d8a978500 rate 20 gen 0 00000100:00100000:3.0:1789762963.028481:0:17485:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762963.028483:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:3.0:1789762963.028487:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_write 00000100:00100000:3.0:1789762963.028491:0:17485:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981d8a978300 rate 20 gen 0 00000100:00100000:3.0:1789762963.028642:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:3.0:1789762963.028646:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:3.0:1789762963.028650:0:17485:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762963.028653:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:3.0:1789762963.028655:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:3.0:1789762963.028658:0:17485:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981d8a978a00 rate 10 gen 0 00000100:00100000:3.0:1789762963.028660:0:17485:0:(nrs_tbf.c:706:nrs_tbf_rule_start()) TBF allowed type: count=0 00000100:00100000:3.0:1789762963.028662:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: uid, str: 500 00000100:00100000:3.0:1789762963.028665:0:17485:0:(nrs_tbf.c:1778:nrs_tbf_expression_parse()) type: 0X7F, dynamic 0, field: opcode, str: ost_read 00000100:00100000:3.0:1789762963.028667:0:17485:0:(nrs_tbf.c:761:nrs_tbf_rule_start()) TBF starts rule@ffff981d8a978600 rate 10 gen 0 00000100:00100000:2.0:1789762963.038415:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.038442:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015147520 00000100:00100000:1.0:1789762963.038451:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015147520 00000100:00080000:3.0:1789762963.038494:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762963.038528:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 366 00000100:00100000:1.0:1789762963.038543:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad82300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:15785:x1876702015147520:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:1.0:1789762963.038563:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762963.038640:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b05b47 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd877 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.038707:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05b4e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.038722:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05b4e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.038736:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05b4e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762963.038771:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762963.038776:0:5805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762963.038796:0:5805:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05b4e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.038822:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b05b47 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd877 expref: 7 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.038841:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05b4e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd877 expref: 6 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.038851:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179000) 00010000:00010000:1.0:1789762963.038855:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762963.038860:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfce00, rc 0) 00000100:00100000:1.0:1789762963.038919:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad82300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15785:x1876702015147520:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 380us (506us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.038928:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 366 00000100:00100000:2.0:1789762963.039884:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.039911:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015147648 00000100:00100000:3.0:1789762963.039916:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015147648 00000100:00100000:2.0:1789762963.039940:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.039950:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015147904 00000100:00100000:3.0:1789762963.039954:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015147904 00000100:00100000:1.0:1789762963.039972:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 367 00000100:00100000:3.0:1789762963.039983:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 368 00000100:00100000:1.0:1789762963.039985:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2364:x1876702015147648:12345-192.168.203.48@tcp:41:mkdir.0 00000004:00000024:1.0:1789762963.040014:0:6611:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 2261647360 avail 2259550208 00000100:00100000:3.0:1789762963.040043:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15785:x1876702015147904:12345-192.168.203.48@tcp:36:mkdir.0 00000100:00100000:1.0:1789762963.040051:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2364:x1876702015147648:12345-192.168.203.48@tcp:41:mkdir.0 Request processed in 71us (168us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.040058:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 367 00010000:00010000:3.0:1789762963.040073:0:5806:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762963.040120:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05b4e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd877 expref: 6 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.040139:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179000) 00010000:00010000:3.0:1789762963.040215:0:5806:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762963.040426:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05b55 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.040439:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05b55 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.040449:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05b55 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.040490:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b5c lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.040499:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b5c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.040507:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b5c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762963.041482:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3d:0x0]: returning size 16384 00010000:00010000:3.0:1789762963.041490:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05b55 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.041496:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05b55 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.041501:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05b55 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.041505:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762963.041509:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05b55 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.041514:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05b55 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.041521:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762963.041523:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762963.041526:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05b55 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.041533:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05b55 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762963.041542:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000054a71e7c reply state 000000007cb5ad10 transno 4294967616 ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b5c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762963.041547:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762963.041589:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15785:x1876702015147904:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 1596us (1649us total) trans 4294967616 rc 0/0 00000100:00100000:3.0:1789762963.041594:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 368 00010000:00010000:1.0:1789762963.042014:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b5c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.042022:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b5c lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.042027:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b5c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.042030:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aa600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762963.042033:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b5c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.042037:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b5c lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.042045:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272cb00) 00010000:00010000:1.0:1789762963.042047:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762963.042049:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b5c lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.042054:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa600/0x416193e012b05b5c lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762963.046168:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.046204:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015148032 00000100:00100000:1.0:1789762963.046211:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015148032 00000100:00100000:1.0:1789762963.046253:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 369 00000100:00100000:1.0:1789762963.046264:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad82680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15786:x1876702015148032:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762963.046276:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762963.046288:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b05b63 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd87e expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.046318:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b05b6a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.046326:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b05b6a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.046332:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b05b6a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762963.046346:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789762963.046349:0:5805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789762963.046358:0:5805:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b05b6a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.046374:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc200/0x416193e012b05b63 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd87e expref: 8 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.046385:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b05b6a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd87e expref: 7 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.046391:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179500) 00010000:00010000:1.0:1789762963.046393:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789762963.046395:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfd200, rc 0) 00000100:00100000:1.0:1789762963.046426:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad82680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15786:x1876702015148032:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 166us (260us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.046431:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 369 00000100:00100000:2.0:1789762963.047284:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.047320:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015148288 00000100:00100000:1.0:1789762963.047326:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015148288 00000100:00100000:1.0:1789762963.047357:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 370 00000100:00100000:1.0:1789762963.047365:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad81f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15786:x1876702015148288:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:1.0:1789762963.047373:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762963.047375:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1789762963.047384:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd200/0x416193e012b05b6a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd87e expref: 7 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047396:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179500) 00010000:00010000:1.0:1789762963.047401:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762963.047409:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05b71 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd885 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047483:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05b78 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047490:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05b78 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047497:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05b78 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047509:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05b7f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047513:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05b7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047518:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05b7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047579:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05b86 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047585:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05b86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047591:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05b86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762963.047597:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3d:0x0]: returning size 16384 00010000:00010000:1.0:1789762963.047606:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05b86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047611:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05b86 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047616:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05b86 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047621:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfc800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762963.047623:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05b86 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047627:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05b86 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047633:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981ec2179400) 00010000:00010000:1.0:1789762963.047635:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762963.047637:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05b86 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047641:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc800/0x416193e012b05b86 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047654:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05b78 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047659:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05b78 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047663:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05b78 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047667:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfde00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762963.047669:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05b78 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047676:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05b78 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047682:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179d00) 00010000:00010000:1.0:1789762963.047684:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762963.047686:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05b78 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047690:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfde00/0x416193e012b05b78 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047697:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05b7f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047701:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05b7f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047705:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05b7f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047708:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87dfcc00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762963.047710:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05b7f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047714:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05b7f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047719:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ec2179e00) 00010000:00010000:1.0:1789762963.047720:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762963.047722:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05b7f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047729:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfcc00/0x416193e012b05b7f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047746:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05b71 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd885 expref: 7 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047752:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ec2179d00) 00010000:00010000:1.0:1789762963.047754:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762963.047757:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfd000/0x416193e012b05b71 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd885 expref: 7 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.047765:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfd000, rc 0) 00000100:00100000:1.0:1789762963.047797:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad81f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15786:x1876702015148288:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 432us (513us total) trans 0 rc 301/301 00000100:00100000:1.0:1789762963.047805:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 370 00000100:00100000:3.0:1789762963.048552:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762963.048591:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015148416 00000100:00100000:0.0:1789762963.048596:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015148416 00000100:00100000:0.0:1789762963.048629:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 371 00000100:00100000:0.0:1789762963.048639:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15786:x1876702015148416:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762963.048647:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762963.048657:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3800/0x416193e012b05b8d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd88c expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.048684:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05b94 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.048691:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05b94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.048698:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05b94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762963.048712:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789762963.048715:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762963.048725:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05b94 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.048738:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3800/0x416193e012b05b8d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd88c expref: 8 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.048750:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05b94 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd88c expref: 7 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.048756:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762963.048758:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762963.048760:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ec21d2a00, rc 0) 00000100:00100000:0.0:1789762963.048794:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15786:x1876702015148416:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 158us (244us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762963.048800:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 371 00000100:00100000:2.0:1789762963.049559:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.049593:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015148672 00000100:00100000:3.0:1789762963.049598:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015148672 00000100:00100000:3.0:1789762963.049625:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 372 00000100:00100000:3.0:1789762963.049632:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad83480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15786:x1876702015148672:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762963.049641:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762963.049651:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b9b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd893 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049679:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05ba2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049687:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05ba2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049693:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05ba2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049702:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ba9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049708:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ba9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049713:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ba9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762963.049751:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3d:0x0]: returning size 16384 00010000:00010000:3.0:1789762963.049794:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05ba2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049799:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05ba2 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049803:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05ba2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049807:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762963.049809:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05ba2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049813:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05ba2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049820:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:3.0:1789762963.049822:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762963.049825:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05ba2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049829:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05ba2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049840:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ba9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049844:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ba9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049848:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ba9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049851:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aa400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762963.049853:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ba9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049857:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ba9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049861:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272c300) 00010000:00010000:3.0:1789762963.049863:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762963.049864:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ba9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049868:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ba9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049889:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b9b lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd893 expref: 9 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049895:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:3.0:1789762963.049896:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762963.049899:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05b9b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd893 expref: 9 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.049905:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aaa00, rc 0) 00000100:00100000:3.0:1789762963.049937:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad83480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:15786:x1876702015148672:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 306us (382us total) trans 4294967617 rc 301/301 00000100:00100000:3.0:1789762963.049942:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 372 00000100:00100000:2.0:1789762963.050692:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.050717:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015148928 00000100:00100000:3.0:1789762963.050722:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015148928 00000100:00100000:3.0:1789762963.050748:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 373 00000100:00100000:3.0:1789762963.050754:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad80700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:15786:x1876702015148928:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:3.0:1789762963.050784:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bb7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.050791:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bb7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.050796:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bb7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762963.050884:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005340a3e6 reply state 00000000c8ed7e86 transno 4294967618 ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bb7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762963.050890:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762963.050895:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3d:0x0]: returning size 16384 00000100:00100000:3.0:1789762963.050932:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad80700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15786:x1876702015148928:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 178us (243us total) trans 4294967618 rc 0/0 00000100:00100000:3.0:1789762963.050938:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 373 00010000:00010000:3.0:1789762963.051427:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bb7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.051437:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bb7 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.051442:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bb7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.051445:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762963.051505:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bb7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.051510:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bb7 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.051520:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e9272c100) 00010000:00010000:3.0:1789762963.051522:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789762963.051525:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bb7 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.051532:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bb7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762963.051567:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.051606:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015149184 00000100:00100000:3.0:1789762963.051611:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015149184 00000100:00100000:3.0:1789762963.051642:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 119 00000100:00100000:3.0:1789762963.051682:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:15786:x1876702015149184:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:3.0:1789762963.051779:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15786:x1876702015149184:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 129us (213us total) trans 4294967619 rc 0/0 00000100:00100000:3.0:1789762963.051785:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 119 00000100:00100000:2.0:1789762963.056111:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762963.056141:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015149312 00000100:00100000:0.0:1789762963.056147:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015149312 00000100:00100000:0.0:1789762963.056191:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 374 00000100:00100000:0.0:1789762963.056200:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15787:x1876702015149312:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:0.0:1789762963.056209:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762963.056219:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3800/0x416193e012b05bbe lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd89a expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056247:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05bc5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056255:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05bc5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056264:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05bc5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056274:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bcc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056280:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bcc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056284:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bcc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056321:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05bd3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056326:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05bd3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056331:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05bd3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762963.056344:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3d:0x0]: returning size 16384 00000004:00200000:0.0:1789762963.056347:0:5806:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789762963.056355:0:5806:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05bd3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056361:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05bc5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056366:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05bc5 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056373:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05bc5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056378:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762963.056380:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05bc5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056384:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05bc5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056390:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:0.0:1789762963.056392:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762963.056394:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05bc5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056398:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05bc5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056405:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bcc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056409:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bcc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056413:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bcc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056417:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762963.056419:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bcc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056425:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bcc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056430:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3900) 00010000:00010000:0.0:1789762963.056431:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762963.056433:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bcc lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056436:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bcc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056452:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3800/0x416193e012b05bbe lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd89a expref: 12 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056460:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05bd3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd89a expref: 11 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.056464:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762963.056466:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789762963.056467:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ec21d2600, rc 0) 00000100:00100000:0.0:1789762963.056504:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15787:x1876702015149312:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 305us (393us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762963.056512:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 374 00000100:00100000:2.0:1789762963.057657:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.057690:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015149568 00000100:00100000:3.0:1789762963.057696:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015149568 00000100:00100000:3.0:1789762963.057729:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 375 00000100:00100000:3.0:1789762963.057737:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15787:x1876702015149568:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:3.0:1789762963.057757:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762963.057767:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05bd3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd89a expref: 11 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.057778:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:3.0:1789762963.057783:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762963.057792:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bda lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.057799:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bda lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.057805:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bda lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762963.057904:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000056178b90 reply state 0000000053a172dc transno 4294967620 ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bda lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762963.057911:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762963.057914:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3d:0x0]: returning size 16384 00000100:00100000:3.0:1789762963.057954:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15787:x1876702015149568:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 219us (299us total) trans 4294967620 rc 0/0 00000100:00100000:3.0:1789762963.057962:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 375 00010000:00010000:1.0:1789762963.058434:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bda lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.058443:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bda lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.058448:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bda lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.058453:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762963.058455:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bda lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.058459:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bda lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.058469:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e9272c100) 00010000:00010000:1.0:1789762963.058471:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789762963.058473:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bda lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.058478:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05bda lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762963.071788:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.071820:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015149696 00000100:00100000:3.0:1789762963.071858:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015149696 00000100:00100000:3.0:1789762963.071890:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 376 00000100:00100000:3.0:1789762963.071896:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15791:x1876702015149696:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762963.071907:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762963.071916:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05be1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8a1 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.071948:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05be8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.071957:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05be8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.071964:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05be8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.071975:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05bef lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.071981:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05bef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.071987:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05bef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072047:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05bf6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072052:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05bf6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072057:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05bf6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762963.072069:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3d:0x0]: returning size 16384 00000004:00200000:3.0:1789762963.072072:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762963.072080:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05bf6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072107:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05be8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072112:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05be8 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072117:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05be8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072121:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aa400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762963.072124:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05be8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072129:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05be8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072137:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:3.0:1789762963.072139:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762963.072143:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05be8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072163:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05be8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072224:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05bef lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072231:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05bef lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072239:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05bef lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072243:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762963.072246:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05bef lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072250:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05bef lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072258:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272c300) 00010000:00010000:3.0:1789762963.072261:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762963.072263:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05bef lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072268:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05bef lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072289:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05be1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8a1 expref: 13 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072302:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05bf6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8a1 expref: 12 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.072307:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e9272c900) 00010000:00010000:3.0:1789762963.072309:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789762963.072310:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880ab200, rc 0) 00000100:00100000:3.0:1789762963.072348:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15791:x1876702015149696:12345-192.168.203.48@tcp:101:dd.500 Request processed in 452us (563us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762963.072358:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 376 00000100:00100000:3.0:1789762963.073315:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762963.073356:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015149952 00000100:00100000:0.0:1789762963.073362:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015149952 00000100:00100000:0.0:1789762963.073403:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 377 00000100:00100000:0.0:1789762963.073416:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15791:x1876702015149952:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:0.0:1789762963.073425:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762963.073434:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bfd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8a8 expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.073504:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05c04 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.073513:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05c04 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.073520:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05c04 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.073568:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05c0b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.073574:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05c0b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.073580:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05c0b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789762963.074286:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x19:0x0], index 0: 0 00000004:00080000:0.0:1789762963.074305:0:5806:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x10:0x0], index 1: 0 00010000:00010000:0.0:1789762963.074491:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05c12 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.074499:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05c12 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.074505:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05c12 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.074530:0:5806:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05c04 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.074536:0:5806:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05c04 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.074541:0:5806:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05c04 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.074545:0:5806:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762963.074548:0:5806:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05c04 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.074553:0:5806:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05c04 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.074562:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e9272c900) 00010000:00010000:0.0:1789762963.074564:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762963.074567:0:5806:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05c04 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.074571:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3200/0x416193e012b05c04 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762963.074584:0:5806:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000012aee5a reply state 00000000f50231ce transno 4294967621 ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05c0b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762963.074589:0:5806:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789762963.074608:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05bfd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8a8 expref: 15 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.074627:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05c12 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8a8 expref: 14 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.074632:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3f:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:0.0:1789762963.074635:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789762963.074637:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ec21d2400, rc 0) 00000100:00100000:0.0:1789762963.074679:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfa680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15791:x1876702015149952:12345-192.168.203.48@tcp:101:dd.500 Request processed in 1268us (1366us total) trans 4294967621 rc 0/0 00000100:00100000:0.0:1789762963.074687:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 377 00010000:00010000:3.0:1789762963.075280:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05c0b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.075288:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05c0b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.075293:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05c0b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.075297:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762963.075301:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05c0b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.075305:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05c0b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.075314:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x77289514 (ffff981ea0aa3900) 00010000:00010000:3.0:1789762963.075317:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762963.075319:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05c0b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.075324:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05c0b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762963.077364:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.077404:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015150080 00000100:00100000:1.0:1789762963.077411:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015150080 00000100:00100000:1.0:1789762963.077461:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 214 00000100:00080000:3.0:1789762963.077469:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762963.077473:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:15791:x1876702015150080:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:1.0:1789762963.077482:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762963.077493:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b05c20 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8af expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762963.077511:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x19:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762963.077522:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=486s v=(0 0) 00010000:00010000:1.0:1789762963.077526:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762963.077533:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b05c20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8af expref: 6 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.077543:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b05c20 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8af expref: 6 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.077551:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfcc00, rc 0) 00000100:00100000:1.0:1789762963.077578:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15791:x1876702015150080:12345-192.168.203.48@tcp:101:dd.500 Request processed in 110us (215us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.077583:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 214 00000100:00100000:0.0:1789762963.085002:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.085072:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015150208 00000100:00100000:2.0:1789762963.085078:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015150208 00002000:00100000:2.0:1789762963.085125:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x19:0x0] (0->1048575) req@ffff981e98e7e680 x1876702015150208/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762963.085134:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7e680 with x1876702015150208 ext(0->1048575) 00002000:00010000:2.0:1789762963.085137:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7e680 00000100:00100000:2.0:1789762963.085163:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452864411574 time 452864531487 state 0x0 ref 1 00000100:00100000:2.0:1789762963.085170:0:12582:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 11: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 452864411574 time 452864531487 state 0x0 ref 1 00000100:00100000:2.0:1789762963.085182:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 3 deadline 486439089793 time 486389089793 state 0x1 ref 1 00000100:00100000:2.0:1789762963.085202:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 2 deadline 486439089793 time 486389111350 state 0x0 ref 1 00000100:00100000:2.0:1789762963.085212:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015150208:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762963.090701:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702015150208:12345-192.168.203.48@tcp:4:dd.500 Request processed in 5489us (5698us total) trans 4294968252 rc 0/0 00000100:00100000:1.0:1789762963.090708:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1357 00000100:00100000:1.0:1789762963.090714:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 2 deadline 486439089793 time 486389111350 state 0x2 ref 0 00002000:00100000:1.0:1789762963.090722:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x19:0x0] (0->1048575) req@ffff981e98e7e680 x1876702015150208/t4294968252(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762963.090729:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7e680 with x1876702015150208 ext(0->1048575) 00002000:00010000:1.0:1789762963.090732:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7e680 00000100:00100000:0.0:1789762963.092466:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.092513:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015150336 00000100:00100000:3.0:1789762963.092521:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015150336 00002000:00100000:3.0:1789762963.092541:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x19:0x0] (1048576->2097151) req@ffff981e98e7ea00 x1876702015150336/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762963.092552:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7ea00 with x1876702015150336 ext(1048576->2097151) 00002000:00010000:3.0:1789762963.092557:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7ea00 00000100:00100000:3.0:1789762963.092569:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 2 deadline 486439089793 time 486389111350 state 0x0 ref 1 00000100:00100000:3.0:1789762963.092582:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 2 deadline 486439111350 time 486389111350 state 0x1 ref 1 00000100:00100000:3.0:1789762963.092606:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 1 deadline 486439111350 time 486396515436 state 0x0 ref 1 00000100:00100000:3.0:1789762963.092624:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702015150336:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762963.095195:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702015150336:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2578us (2728us total) trans 4294968253 rc 0/0 00000100:00100000:3.0:1789762963.095205:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1358 00000100:00100000:3.0:1789762963.095213:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 1 deadline 486439111350 time 486396515436 state 0x2 ref 0 00002000:00100000:3.0:1789762963.095221:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x19:0x0] (1048576->2097151) req@ffff981e98e7ea00 x1876702015150336/t4294968253(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762963.095229:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7ea00 with x1876702015150336 ext(1048576->2097151) 00002000:00010000:3.0:1789762963.095232:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7ea00 00000100:00100000:0.0:1789762963.099391:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.099470:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015150464 00000100:00100000:2.0:1789762963.099476:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015150464 00002000:00100000:2.0:1789762963.099502:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x19:0x0] (2097152->3145727) req@ffff981e98e7fb80 x1876702015150464/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762963.099513:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7fb80 with x1876702015150464 ext(2097152->3145727) 00002000:00010000:2.0:1789762963.099518:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7fb80 00000100:00100000:2.0:1789762963.099530:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 1 deadline 486439111350 time 486396515436 state 0x0 ref 1 00000100:00100000:2.0:1789762963.099553:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 1 deadline 486446515436 time 486396515436 state 0x1 ref 1 00000100:00100000:2.0:1789762963.099584:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486446515436 time 486403492184 state 0x0 ref 1 00000100:00100000:2.0:1789762963.099599:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2364:x1876702015150464:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762963.103030:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2364:x1876702015150464:12345-192.168.203.48@tcp:4:dd.500 Request processed in 3428us (3632us total) trans 4294968254 rc 0/0 00000100:00100000:3.0:1789762963.103045:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1359 00000100:00100000:3.0:1789762963.103057:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486446515436 time 486403492184 state 0x2 ref 0 00002000:00100000:3.0:1789762963.103073:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x19:0x0] (2097152->3145727) req@ffff981e98e7fb80 x1876702015150464/t4294968254(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762963.103080:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981e98e7fb80 with x1876702015150464 ext(2097152->3145727) 00002000:00010000:3.0:1789762963.103101:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981e98e7fb80 00000100:00100000:2.0:1789762963.107324:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.107353:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.107373:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015150720 00000100:00100000:1.0:1789762963.107382:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015150720 00000100:00100000:0.0:1789762963.107394:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015150592 00000100:00100000:0.0:1789762963.107397:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015150592 00000100:00100000:1.0:1789762963.107420:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 215 00000100:00100000:1.0:1789762963.107429:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0+-99:2362:x1876702015150720:12345-192.168.203.48@tcp:8:dd.500 00002000:00100000:0.0:1789762963.107435:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x19:0x0] (3145728->4194303) req@ffff981ebfdc9f80 x1876702015150592/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762963.107445:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc9f80 with x1876702015150592 ext(3145728->4194303) 00002000:00010000:0.0:1789762963.107449:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdc9f80 00010000:00080000:1.0:1789762963.107453:0:6648:0:(ldlm_lib.c:1444:target_handle_connect()) lustre-OST0001: connection from f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp t0 exp 0000000000000000 cur 486 last 0 00000100:00100000:0.0:1789762963.107461:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486446515436 time 486403492184 state 0x0 ref 1 00000100:00100000:0.0:1789762963.107474:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486453492184 time 486403492184 state 0x1 ref 1 00000020:00000080:1.0:1789762963.107478:0:6648:0:(genops.c:1385:class_connect()) connect: client f1bd28de-ee78-4d18-bad3-fa7cec968893, cookie 0x416193e012b05c27 00002000:00100000:1.0:1789762963.107489:0:6648:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli f1bd28de-ee78-4d18-bad3-fa7cec968893/ffff981d8a641800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 211396a ocd_grant: 30670848 ocd_index: 0 ocd_group 0 00002000:00100000:1.0:1789762963.107499:0:6648:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:1.0:1789762963.107639:0:6648:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:1.0:1789762963.107670:0:6648:0:(import.c:65:import_set_state_nolock()) ffff981d8a644800 : changing import state from RECOVER to FULL 00000100:00100000:1.0:1789762963.107706:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2362:x1876702015150720:12345-192.168.203.48@tcp:8:dd.500 Request processed in 277us (384us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.107715:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 215 00000100:00100000:0.0:1789762963.108379:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.108427:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015150848 00000100:00100000:1.0:1789762963.108432:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015150848 00000100:00100000:1.0:1789762963.108467:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 216 00000100:00100000:1.0:1789762963.108474:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e98e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:15791:x1876702015150848:12345-192.168.203.48@tcp:101:dd.500 00000100:00080000:3.0:1789762963.108476:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1789762963.108483:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762963.108498:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002b972c92/0x416193e012b05c2e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8b6 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762963.108517:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x10:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762963.108530:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=486s v=(0 0) 00010000:00010000:1.0:1789762963.108534:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762963.108541:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002b972c92/0x416193e012b05c2e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8b6 expref: 7 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.108554:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002b972c92/0x416193e012b05c2e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8b6 expref: 7 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.108563:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfde00, rc 0) 00000100:00100000:1.0:1789762963.108587:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e98e7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:15791:x1876702015150848:12345-192.168.203.48@tcp:101:dd.500 Request processed in 114us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.108593:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 216 00000100:00100000:0.0:1789762963.116618:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.116671:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015150976 00000100:00100000:1.0:1789762963.116676:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015150976 00002000:00100000:1.0:1789762963.116693:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_008: refresh rw locks for [0x280000400:0x10:0x0] (0->1048575) req@ffff981eadb18700 x1876702015150976/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762963.116701:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eadb18700 with x1876702015150976 ext(0->1048575) 00002000:00010000:1.0:1789762963.116704:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eadb18700 00000100:00100000:1.0:1789762963.116714:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486453492184 time 486403492184 state 0x1 ref 2 00000100:00100000:0.0:1789762963.123345:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.123399:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015151104 00000100:00100000:1.0:1789762963.123406:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015151104 00002000:00100000:1.0:1789762963.123426:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_005: refresh rw locks for [0x280000400:0x10:0x0] (1048576->2097151) req@ffff981eadb18e00 x1876702015151104/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762963.123433:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eadb18e00 with x1876702015151104 ext(1048576->2097151) 00002000:00010000:1.0:1789762963.123436:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eadb18e00 00000100:00100000:1.0:1789762963.123445:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486453492184 time 486403492184 state 0x1 ref 3 00000100:00100000:3.0:1789762963.149665:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486503565383 time 486453565383 state 0x1 ref 3 00000100:00100000:3.0:1789762963.149684:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:2363:x1876702015150592:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762963.152400:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:2363:x1876702015150592:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2719us (45043us total) trans 4294968255 rc 0/0 00000100:00100000:3.0:1789762963.152411:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1360 00002000:00100000:3.0:1789762963.152424:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x19:0x0] (3145728->4194303) req@ffff981ebfdc9f80 x1876702015150592/t4294968255(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762963.152434:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc9f80 with x1876702015150592 ext(3145728->4194303) 00002000:00010000:3.0:1789762963.152440:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdc9f80 00000100:00100000:0.0:1789762963.199747:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486553645271 time 486503645271 state 0x1 ref 2 00000100:00100000:0.0:1789762963.199765:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2364:x1876702015150976:12345-192.168.203.48@tcp:4:dd.500 00000001:00100000:2.0:1789762963.210361:0:7605:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of f1bd28de-ee78-4d18-bad3-fa7cec968893 00000001:00080000:2.0:1789762963.220190:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967435 is committed 00000100:00100000:3.0:1789762963.220233:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+7:2364:x1876702015150976:12345-192.168.203.48@tcp:4:dd.500 Request processed in 20469us (103611us total) trans 4294967435 rc 0/0 00000100:00100000:3.0:1789762963.220242:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1361 00002000:00100000:3.0:1789762963.220252:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x10:0x0] (0->1048575) req@ffff981eadb18700 x1876702015150976/t4294967435(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762963.220260:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eadb18700 with x1876702015150976 ext(0->1048575) 00002000:00010000:3.0:1789762963.220265:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eadb18700 00000100:00100000:1.0:1789762963.222343:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.222385:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015151232 00000100:00100000:3.0:1789762963.222392:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015151232 00002000:00100000:3.0:1789762963.222420:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x10:0x0] (2097152->3145727) req@ffff981d81ea4700 x1876702015151232/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762963.222427:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81ea4700 with x1876702015151232 ext(2097152->3145727) 00002000:00010000:3.0:1789762963.222430:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81ea4700 00000100:00100000:3.0:1789762963.222443:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486553645271 time 486503645271 state 0x1 ref 2 00000100:00100000:2.0:1789762963.228570:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.228612:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015151360 00000100:00100000:1.0:1789762963.228618:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015151360 00002000:00100000:1.0:1789762963.228638:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x10:0x0] (3145728->4194303) req@ffff981ebfdcbb80 x1876702015151360/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762963.228647:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdcbb80 with x1876702015151360 ext(3145728->4194303) 00002000:00010000:1.0:1789762963.228650:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdcbb80 00000100:00100000:1.0:1789762963.228659:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486553645271 time 486503645271 state 0x1 ref 3 00000100:00100000:2.0:1789762963.235344:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762963.235389:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015151488 00000100:00100000:0.0:1789762963.235394:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015151488 00002000:00100000:0.0:1789762963.235415:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x19:0x0] (4194304->5242879) req@ffff981ebfdcaa00 x1876702015151488/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762963.235422:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdcaa00 with x1876702015151488 ext(4194304->5242879) 00002000:00010000:0.0:1789762963.235426:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdcaa00 00000100:00100000:0.0:1789762963.235435:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486553645271 time 486503645271 state 0x1 ref 4 00000100:00100000:1.0:1789762963.241811:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.241870:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015151616 00000100:00100000:2.0:1789762963.241876:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015151616 00002000:00100000:2.0:1789762963.241899:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x240000400:0x19:0x0] (5242880->6291455) req@ffff981d81ea7b80 x1876702015151616/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762963.241908:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81ea7b80 with x1876702015151616 ext(5242880->6291455) 00002000:00010000:2.0:1789762963.241911:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81ea7b80 00000100:00100000:2.0:1789762963.241921:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486553645271 time 486503645271 state 0x1 ref 5 00000100:00100000:2.0:1789762963.248109:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.248163:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015151744 00000100:00100000:3.0:1789762963.248169:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015151744 00002000:00100000:3.0:1789762963.248189:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x19:0x0] (6291456->7340031) req@ffff981ebfdc8380 x1876702015151744/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762963.248198:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc8380 with x1876702015151744 ext(6291456->7340031) 00002000:00010000:3.0:1789762963.248201:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdc8380 00000100:00100000:3.0:1789762963.248210:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486553645271 time 486503645271 state 0x1 ref 6 00000100:00100000:0.0:1789762963.249805:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486603710425 time 486553710425 state 0x1 ref 6 00000100:00100000:0.0:1789762963.249818:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb18e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2363:x1876702015151104:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762963.252359:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb18e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:2363:x1876702015151104:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2539us (129011us total) trans 4294967436 rc 0/0 00000100:00100000:1.0:1789762963.252367:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1362 00002000:00100000:1.0:1789762963.252374:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x10:0x0] (1048576->2097151) req@ffff981eadb18e00 x1876702015151104/t4294967436(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762963.252387:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eadb18e00 with x1876702015151104 ext(1048576->2097151) 00002000:00010000:1.0:1789762963.252391:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eadb18e00 00000100:00100000:2.0:1789762963.254672:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.254723:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015151872 00000100:00100000:1.0:1789762963.254728:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015151872 00002000:00100000:1.0:1789762963.254749:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x19:0x0] (7340032->8388607) req@ffff981ebfdca680 x1876702015151872/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762963.254757:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdca680 with x1876702015151872 ext(7340032->8388607) 00002000:00010000:1.0:1789762963.254760:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdca680 00000100:00100000:1.0:1789762963.254771:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486603710425 time 486553710425 state 0x1 ref 6 00000100:00100000:0.0:1789762963.261071:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.261135:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015152000 00000100:00100000:2.0:1789762963.261141:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015152000 00002000:00100000:2.0:1789762963.261176:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0x10:0x0] (4194304->5242879) req@ffff981eadb19f80 x1876702015152000/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762963.261190:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eadb19f80 with x1876702015152000 ext(4194304->5242879) 00002000:00010000:2.0:1789762963.261193:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eadb19f80 00000100:00100000:2.0:1789762963.261203:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486603710425 time 486553710425 state 0x1 ref 7 00000100:00100000:0.0:1789762963.267367:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.267418:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015152128 00000100:00100000:2.0:1789762963.267424:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015152128 00002000:00100000:2.0:1789762963.267443:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0x10:0x0] (5242880->6291455) req@ffff981eadb1a680 x1876702015152128/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762963.267451:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eadb1a680 with x1876702015152128 ext(5242880->6291455) 00002000:00010000:2.0:1789762963.267454:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eadb1a680 00000100:00100000:2.0:1789762963.267463:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486603710425 time 486553710425 state 0x1 ref 8 00000100:00100000:3.0:1789762963.299904:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486653805541 time 486603805541 state 0x1 ref 8 00000100:00100000:3.0:1789762963.299920:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:2366:x1876702015151232:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762963.302632:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:2366:x1876702015151232:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2714us (80288us total) trans 4294967437 rc 0/0 00000100:00100000:3.0:1789762963.302639:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1363 00002000:00100000:3.0:1789762963.302646:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x10:0x0] (2097152->3145727) req@ffff981d81ea4700 x1876702015151232/t4294967437(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762963.302654:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81ea4700 with x1876702015151232 ext(2097152->3145727) 00002000:00010000:3.0:1789762963.302659:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81ea4700 00000100:00100000:0.0:1789762963.304550:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.304602:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015152256 00000100:00100000:2.0:1789762963.304608:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015152256 00002000:00100000:2.0:1789762963.304630:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x10:0x0] (6291456->7340031) req@ffff981eadb1ad80 x1876702015152256/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762963.304638:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eadb1ad80 with x1876702015152256 ext(6291456->7340031) 00002000:00010000:2.0:1789762963.304641:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eadb1ad80 00000100:00100000:2.0:1789762963.304653:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486653805541 time 486603805541 state 0x1 ref 8 00000100:00100000:2.0:1789762963.311297:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.311330:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.311337:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015152384 00000100:00100000:1.0:1789762963.311343:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015152384 00002000:00100000:1.0:1789762963.311364:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x10:0x0] (7340032->8388607) req@ffff981ebfdc8000 x1876702015152384/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762963.311372:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc8000 with x1876702015152384 ext(7340032->8388607) 00002000:00010000:1.0:1789762963.311375:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdc8000 00000100:00100000:1.0:1789762963.311384:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486653805541 time 486603805541 state 0x1 ref 9 00000100:00100000:3.0:1789762963.311411:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015152640 00000100:00100000:3.0:1789762963.311417:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015152640 00000100:00100000:3.0:1789762963.311452:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 120 00000100:00100000:3.0:1789762963.311460:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15791:x1876702015152640:12345-192.168.203.48@tcp:35:dd.500 00000100:00100000:3.0:1789762963.311650:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15791:x1876702015152640:12345-192.168.203.48@tcp:35:dd.500 Request processed in 191us (319us total) trans 4294967622 rc 0/0 00000100:00100000:3.0:1789762963.311656:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 120 00000100:00100000:0.0:1789762963.343388:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.343429:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015152896 00000100:00100000:3.0:1789762963.343436:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015152896 00000100:00100000:3.0:1789762963.343476:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 378 00000100:00100000:3.0:1789762963.343490:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15802:x1876702015152896:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762963.343511:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762963.343530:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05c35 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8bd expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.343617:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05c3c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.343629:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05c3c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.343637:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05c3c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.343688:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05c43 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.343694:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05c43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.343699:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05c43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762963.344057:0:5805:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x11:0x0], index 1: 0 00000004:00080000:3.0:1789762963.344062:0:5805:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1a:0x0], index 0: 0 00010000:00010000:3.0:1789762963.344205:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05c4a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.344213:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05c4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.344218:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05c4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.344242:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05c3c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.344247:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05c3c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.344252:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05c3c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.344256:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762963.344259:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05c3c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.344264:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05c3c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.344273:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e9272c900) 00010000:00010000:3.0:1789762963.344275:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762963.344279:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05c3c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.344283:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05c3c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762963.344291:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000f157730 reply state 0000000020a31f45 transno 4294967623 ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05c43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762963.344297:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789762963.344316:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05c35 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8bd expref: 18 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.344325:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05c4a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8bd expref: 17 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.344330:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762963.344332:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762963.344334:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880abe00, rc 0) 00000100:00100000:3.0:1789762963.344375:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15802:x1876702015152896:12345-192.168.203.48@tcp:101:dd.0 Request processed in 891us (993us total) trans 4294967623 rc 0/0 00000100:00100000:3.0:1789762963.344381:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 378 00010000:00010000:1.0:1789762963.344953:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05c43 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.344962:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05c43 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.344967:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05c43 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.344972:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aa400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762963.344974:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05c43 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.344979:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05c43 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.344986:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x992ad74f (ffff981e9272c500) 00010000:00010000:1.0:1789762963.344988:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762963.344990:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05c43 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.344994:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05c43 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762963.349210:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.349242:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015153024 00000100:00100000:1.0:1789762963.349247:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015153024 00000100:00100000:1.0:1789762963.349278:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 217 00000100:00100000:1.0:1789762963.349284:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15802:x1876702015153024:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762963.349292:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762963.349303:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8c4 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762963.349320:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x11:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762963.349332:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=486s v=(0 0) 00010000:00010000:1.0:1789762963.349336:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762963.349342:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 16 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.349357:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 16 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.349364:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc200, rc 0) 00000100:00100000:1.0:1789762963.349391:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:15802:x1876702015153024:12345-192.168.203.48@tcp:101:dd.0 Request processed in 108us (183us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.349397:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 217 00000100:00100000:0.0:1789762963.349646:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.349679:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015153152 00000100:00100000:1.0:1789762963.349684:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015153152 00000100:00100000:1.0:1789762963.349710:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 218 00000100:00100000:1.0:1789762963.349715:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+16:15803:x1876702015153152:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762963.349728:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762963.349736:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004887b6d3/0x416193e012b05c5f lrc: 2/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8cb expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.349752:0:6648:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 17 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762963.349762:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:1.0:1789762963.349765:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762963.349769:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789762963.349773:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004887b6d3/0x416193e012b05c5f lrc: 4/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8cb expref: 17 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.349802:0:6648:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 17 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762963.349804:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015153280 00000100:00100000:2.0:1789762963.349811:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015153280 00010000:00010000:1.0:1789762963.349821:0:6648:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 17 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.349834:0:6648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81ea6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6648:1876702022200448:192.168.203.48@tcp:104: 00000100:00100000:2.0:1789762963.349855:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 219 00000100:00100000:2.0:1789762963.349866:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15804:x1876702015153280:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:2.0:1789762963.349874:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762963.349883:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c1889e1e/0x416193e012b05c66 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8d2 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.349900:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x11:0x0].0x0 since=0s v=(1 0) 00000100:00100000:1.0:1789762963.349902:0:6648:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e878baa80 going to sleep for 11 seconds 00010000:00010000:2.0:1789762963.349903:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(2 0) 00010000:00010000:2.0:1789762963.349911:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c1889e1e/0x416193e012b05c66 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d2 expref: 19 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.349921:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c1889e1e/0x416193e012b05c66 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d2 expref: 19 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.349934:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfc92c00, rc 0) 00000100:00100000:3.0:1789762963.349944:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486703844390 time 486653844390 state 0x1 ref 9 00000100:00100000:3.0:1789762963.349958:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:2365:x1876702015151360:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762963.349983:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:15804:x1876702015153280:12345-192.168.203.48@tcp:101:dd.0 Request processed in 117us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762963.349992:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 219 00000100:00100000:1.0:1789762963.350759:0:6648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81ea6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6648:1876702022200448:192.168.203.48@tcp:104: 00010000:00010000:1.0:1789762963.350780:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004887b6d3/0x416193e012b05c5f lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8cb expref: 19 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.350788:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc400, rc 0) 00000100:00100000:1.0:1789762963.350816:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:15803:x1876702015153152:12345-192.168.203.48@tcp:101:dd.0 Request processed in 1100us (1170us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.350824:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 218 00000100:00100000:3.0:1789762963.352823:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:2365:x1876702015151360:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2867us (124252us total) trans 4294967438 rc 0/0 00000100:00100000:3.0:1789762963.352830:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1364 00002000:00100000:3.0:1789762963.352838:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0x10:0x0] (3145728->4194303) req@ffff981ebfdcbb80 x1876702015151360/t4294967438(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762963.352845:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdcbb80 with x1876702015151360 ext(3145728->4194303) 00002000:00010000:3.0:1789762963.352849:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdcbb80 00000100:00100000:2.0:1789762963.353890:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.353931:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015153408 00000100:00100000:1.0:1789762963.353937:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015153408 00000100:00100000:1.0:1789762963.353966:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 220 00000100:00100000:1.0:1789762963.353973:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:15805:x1876702015153408:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762963.353981:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762963.353989:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05c6d lrc: 2/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 5 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8d9 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.354003:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x11:0x0].0x0 since=0s v=(2 0) 00010000:00010000:1.0:1789762963.354007:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(3 0) 00010000:00010000:1.0:1789762963.354011:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05c6d lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d9 expref: 20 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.354019:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05c6d lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 5 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d9 expref: 20 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.354026:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfd400, rc 0) 00000100:00100000:1.0:1789762963.354054:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:15805:x1876702015153408:12345-192.168.203.48@tcp:101:dd.0 Request processed in 81us (163us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.354061:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 220 00000100:00100000:0.0:1789762963.354875:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.354918:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015153536 00000100:00100000:3.0:1789762963.354925:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015153536 00000100:00100000:3.0:1789762963.354954:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 221 00000100:00100000:3.0:1789762963.354960:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15807:x1876702015153536:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762963.354967:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762963.354979:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05c74 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8e0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762963.354994:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1a:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789762963.355003:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=486s v=(0 0) 00010000:00010000:3.0:1789762963.355006:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762963.355011:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05c74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8e0 expref: 19 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.355020:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05c74 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8e0 expref: 19 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.355027:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aaa00, rc 0) 00000100:00100000:3.0:1789762963.355053:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:15807:x1876702015153536:12345-192.168.203.48@tcp:101:dd.0 Request processed in 94us (179us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762963.355059:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 221 00000100:00100000:0.0:1789762963.371451:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.371506:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015153664 00000100:00100000:1.0:1789762963.371513:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015153664 00000100:00100000:1.0:1789762963.371554:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 222 00000100:00100000:1.0:1789762963.371573:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:15813:x1876702015153664:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762963.371584:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762963.371596:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ceb8603d/0x416193e012b05c7b lrc: 2/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 6 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8e7 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.371617:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x11:0x0].0x0 since=0s v=(3 0) 00010000:00010000:1.0:1789762963.371622:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(4 0) 00010000:00010000:1.0:1789762963.371628:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ceb8603d/0x416193e012b05c7b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 6 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8e7 expref: 21 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.371639:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000ceb8603d/0x416193e012b05c7b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 6 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8e7 expref: 21 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.371650:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc800, rc 0) 00000100:00100000:1.0:1789762963.371691:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:15813:x1876702015153664:12345-192.168.203.48@tcp:101:dd.0 Request processed in 128us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.371699:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 222 00000100:00100000:2.0:1789762963.377032:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.377082:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.377107:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015153792 00000100:00100000:1.0:1789762963.377113:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015153792 00000100:00100000:3.0:1789762963.377133:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015153920 00002000:00100000:1.0:1789762963.377139:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x1a:0x0] (1048576->2097151) req@ffff981ebfdc8700 x1876702015153792/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.377164:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc8700 with x1876702015153792 ext(1048576->2097151) 00002000:00010000:1.0:1789762963.377168:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdc8700 00000100:00100000:1.0:1789762963.377181:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431148638436 time 431151141512 state 0x0 ref 1 00000100:00100000:1.0:1789762963.377188:0:12582:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431148638436 time 431151141512 state 0x0 ref 1 00000100:00100000:3.0:1789762963.377199:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015153920 00000100:00100000:1.0:1789762963.377223:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431151241512 time 431151141512 state 0x1 ref 1 00000100:00100000:3.0:1789762963.377237:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 223 00000100:00100000:3.0:1789762963.377247:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:15810:x1876702015153920:12345-192.168.203.48@tcp:101:dd.0 00000100:00100000:1.0:1789762963.377258:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431151241512 time 486681159506 state 0x0 ref 1 00010000:00010000:3.0:1789762963.377261:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1789762963.377272:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+19:2365:x1876702015153792:12345-192.168.203.48@tcp:4:dd.0 00010000:00010000:3.0:1789762963.377272:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 7 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8ee expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.377294:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x11:0x0].0x0 since=0s v=(4 0) 00010000:00010000:3.0:1789762963.377298:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(5 0) 00010000:00010000:3.0:1789762963.377303:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 7 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 22 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.377313:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 7 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 22 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.377321:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880ab000, rc 0) 00000100:00100000:3.0:1789762963.377354:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:15810:x1876702015153920:12345-192.168.203.48@tcp:101:dd.0 Request processed in 110us (273us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762963.377360:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 223 00000100:00100000:0.0:1789762963.377490:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.377521:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015154048 00000100:00100000:3.0:1789762963.377526:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015154048 00000100:00100000:3.0:1789762963.377561:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 224 00000100:00100000:3.0:1789762963.377573:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:15812:x1876702015154048:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762963.377583:0:6881:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762963.377594:0:6881:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c462886a/0x416193e012b05c89 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8f5 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.377611:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x11:0x0].0x0 since=0s v=(5 0) 00010000:00010000:3.0:1789762963.377614:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:3.0:1789762963.377619:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c462886a/0x416193e012b05c89 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8f5 expref: 23 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.377628:0:6881:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c462886a/0x416193e012b05c89 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8f5 expref: 23 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.377635:0:6881:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aae00, rc 0) 00000100:00100000:3.0:1789762963.377667:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15812:x1876702015154048:12345-192.168.203.48@tcp:101:dd.0 Request processed in 98us (179us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762963.377673:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 224 00000100:00100000:1.0:1789762963.381668:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+21:2365:x1876702015153792:12345-192.168.203.48@tcp:4:dd.0 Request processed in 4398us (4635us total) trans 4294968256 rc 0/0 00000100:00100000:1.0:1789762963.381676:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1373 00000100:00100000:1.0:1789762963.381683:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431151241512 time 486681159506 state 0x2 ref 0 00002000:00100000:1.0:1789762963.381692:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x1a:0x0] (1048576->2097151) req@ffff981ebfdc8700 x1876702015153792/t4294968256(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.381700:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc8700 with x1876702015153792 ext(1048576->2097151) 00002000:00010000:1.0:1789762963.381704:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdc8700 00000100:00100000:0.0:1789762963.381778:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.381865:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015154176 00000100:00100000:3.0:1789762963.381870:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015154176 00002000:00100000:3.0:1789762963.381891:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x11:0x0] (0->1048575) req@ffff981ebfdbb800 x1876702015154176/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762963.381902:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbb800 with x1876702015154176 ext(0->1048575) 00010000:00010000:3.0:1789762963.381909:0:6654:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 23 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.381925:0:6654:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 23 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762963.381937:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981ebfdbb800 00000100:00100000:3.0:1789762963.381946:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 430962555255 time 431067755888 state 0x0 ref 1 00000100:00100000:3.0:1789762963.381952:0:6654:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 430962555255 time 431067755888 state 0x0 ref 1 00000100:00100000:3.0:1789762963.381960:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 431067855888 time 431067755888 state 0x1 ref 1 00000100:00100000:3.0:1789762963.381979:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 431067855888 time 486685889025 state 0x0 ref 1 00000100:00100000:3.0:1789762963.381988:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2365:x1876702015154176:12345-192.168.203.48@tcp:4:dd.0 00010000:00010000:1.0:1789762963.383210:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762963.386310:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2365:x1876702015154176:12345-192.168.203.48@tcp:4:dd.0 Request processed in 4322us (4532us total) trans 4294967439 rc 0/0 00000100:00100000:3.0:1789762963.386316:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 8 00000100:00100000:3.0:1789762963.386321:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 431067855888 time 486685889025 state 0x2 ref 0 00002000:00100000:3.0:1789762963.386329:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x11:0x0] (0->1048575) req@ffff981ebfdbb800 x1876702015154176/t4294967439(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762963.386337:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbb800 with x1876702015154176 ext(0->1048575) 00010000:00010000:3.0:1789762963.386343:0:6654:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 25 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.386354:0:6654:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 25 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762963.386361:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981ebfdbb800 00000100:00100000:0.0:1789762963.387871:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.387914:0:7595:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015154304 00000100:00100000:1.0:1789762963.387919:0:7595:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015154304 00010000:00010000:1.0:1789762963.387938:0:7595:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 25 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.387963:0:7595:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 65 00000100:00100000:1.0:1789762963.387970:0:7595:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2366:x1876702015154304:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762963.387985:0:7595:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1789762963.387995:0:7595:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:1.0:1789762963.387997:0:7595:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb mtime from disk: 0 -> 1789762963 00002000:00010000:1.0:1789762963.387999:0:7595:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb ctime from disk: 0 -> 1789762963 00010000:00010000:1.0:1789762963.388003:0:7595:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 25 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388013:0:7595:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 25 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388028:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c58 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 9 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8c4 expref: 25 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388038:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388041:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388047:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004887b6d3/0x416193e012b05c5f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8cb expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388054:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004887b6d3/0x416193e012b05c5f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8cb expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388059:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388061:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388065:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c1889e1e/0x416193e012b05c66 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d2 expref: 24 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388069:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c1889e1e/0x416193e012b05c66 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d2 expref: 24 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388073:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388075:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388078:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05c6d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d9 expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388083:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05c6d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d9 expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388117:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388119:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388123:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ceb8603d/0x416193e012b05c7b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8e7 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388135:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ceb8603d/0x416193e012b05c7b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8e7 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388140:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388142:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388145:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388164:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388168:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388170:0:7595:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.388173:0:7595:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c462886a/0x416193e012b05c89 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8f5 expref: 24 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388179:0:7595:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c462886a/0x416193e012b05c89 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8f5 expref: 24 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.388202:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c462886a/0x416193e012b05c89 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8f5 expref: 24 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.388211:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81ea6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022200576:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762963.388300:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.388309:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81ea4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022200704:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762963.388337:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000ceb8603d/0x416193e012b05c7b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8e7 expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.388346:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022200832:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762963.388374:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05c6d lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d9 expref: 24 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.388381:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebc49ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022200960:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762963.388403:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c1889e1e/0x416193e012b05c66 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d2 expref: 24 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.388410:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022201088:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762963.388428:0:7595:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000004887b6d3/0x416193e012b05c5f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8cb expref: 24 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.388436:0:7595:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1c700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022201216:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762963.388457:0:7595:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e878baa80 going to sleep for 11 seconds 00000100:00100000:1.0:1789762963.389918:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81ea6a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022200576:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762963.389926:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81ea4e00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022200704:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762963.389931:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49ca80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022200832:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762963.389937:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebc49ea00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022200960:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762963.389942:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1f480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022201088:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762963.390197:0:7595:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1c700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:7595:1876702022201216:192.168.203.48@tcp:105: 00010000:00010000:1.0:1789762963.390222:0:7595:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762963.390258:0:7595:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2366:x1876702015154304:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 2288us (2388us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.390267:0:7595:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 65 00000100:00100000:0.0:1789762963.396882:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.396923:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015154432 00000100:00100000:1.0:1789762963.396929:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015154432 00000100:00100000:1.0:1789762963.396957:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 225 00000100:00100000:1.0:1789762963.396965:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:15811:x1876702015154432:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762963.396973:0:6648:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762963.396981:0:6648:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c90 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd8fc expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.396997:0:6648:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 25 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.397003:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x11:0x0].0x0 since=0s v=(6 0) 00010000:00010000:1.0:1789762963.397006:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762963.397009:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c90 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8fc expref: 25 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.397027:0:6648:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x50000000020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 25 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.397041:0:6648:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x70000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 25 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.397049:0:6648:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6648:1876702022201344:192.168.203.48@tcp:104: 00000100:00100000:1.0:1789762963.397084:0:6648:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e878baa80 going to sleep for 11 seconds 00000100:00100000:1.0:1789762963.397936:0:6648:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6648:1876702022201344:192.168.203.48@tcp:104: 00010000:00010000:1.0:1789762963.397948:0:6648:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c90 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8fc expref: 25 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.397957:0:6648:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc200, rc 0) 00000100:00100000:1.0:1789762963.397982:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:15811:x1876702015154432:12345-192.168.203.48@tcp:101:dd.0 Request processed in 1019us (1101us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.397988:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 225 00000100:00100000:1.0:1789762963.400035:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486753939848 time 486703939848 state 0x1 ref 8 00000100:00100000:1.0:1789762963.400047:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:2365:x1876702015151488:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762963.402616:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:2365:x1876702015151488:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2569us (167271us total) trans 4294968257 rc 0/0 00000100:00100000:1.0:1789762963.402624:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1365 00002000:00100000:1.0:1789762963.402632:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x19:0x0] (4194304->5242879) req@ffff981ebfdcaa00 x1876702015151488/t4294968257(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762963.402639:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdcaa00 with x1876702015151488 ext(4194304->5242879) 00002000:00010000:1.0:1789762963.402644:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdcaa00 00000100:00100000:2.0:1789762963.427084:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.427173:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015154560 00000100:00100000:1.0:1789762963.427179:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015154560 00002000:00100000:1.0:1789762963.427202:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x1a:0x0] (0->1048575) req@ffff981ebfdc8a80 x1876702015154560/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.427210:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc8a80 with x1876702015154560 ext(0->1048575) 00002000:00010000:1.0:1789762963.427213:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdc8a80 00000100:00100000:1.0:1789762963.427223:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431151241512 time 486681159506 state 0x0 ref 1 00000100:00100000:1.0:1789762963.427230:0:12581:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 431151241512 time 486681159506 state 0x0 ref 1 00000100:00100000:1.0:1789762963.427260:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486681259506 time 486681159506 state 0x1 ref 1 00000100:00100000:1.0:1789762963.427278:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486681259506 time 486731187382 state 0x0 ref 1 00000100:00100000:1.0:1789762963.427287:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:2365:x1876702015154560:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762963.429587:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:2365:x1876702015154560:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2299us (2505us total) trans 4294968258 rc 0/0 00000100:00100000:1.0:1789762963.429594:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1374 00000100:00100000:1.0:1789762963.429600:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486681259506 time 486731187382 state 0x2 ref 0 00002000:00100000:1.0:1789762963.429609:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x1a:0x0] (0->1048575) req@ffff981ebfdc8a80 x1876702015154560/t4294968258(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.429616:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc8a80 with x1876702015154560 ext(0->1048575) 00002000:00010000:1.0:1789762963.429620:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdc8a80 00000100:00100000:2.0:1789762963.434605:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762963.434690:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015154688 00000100:00100000:0.0:1789762963.434700:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015154688 00002000:00100000:0.0:1789762963.434739:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0x11:0x0] (4194304->5242879) req@ffff981ebfdc9c00 x1876702015154688/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762963.434752:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc9c00 with x1876702015154688 ext(4194304->5242879) 00010000:00010000:0.0:1789762963.434762:0:14912:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 25 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.434787:0:14912:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 25 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789762963.434797:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981ebfdc9c00 00000100:00100000:0.0:1789762963.434813:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 431067855888 time 486685889025 state 0x0 ref 1 00000100:00100000:0.0:1789762963.434823:0:14912:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 431067855888 time 486685889025 state 0x0 ref 1 00000100:00100000:0.0:1789762963.434840:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 486685989025 time 486685889025 state 0x1 ref 1 00000100:00100000:0.0:1789762963.434866:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 486685989025 time 486738775284 state 0x0 ref 1 00000100:00100000:0.0:1789762963.434876:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2366:x1876702015154688:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762963.437882:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2366:x1876702015154688:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2999us (3275us total) trans 4294967440 rc 0/0 00000100:00100000:1.0:1789762963.437897:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 9 00000100:00100000:1.0:1789762963.437909:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebc2bfe00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804900 gen 0 rate 10000 depth 3 token 2 deadline 486685989025 time 486738775284 state 0x2 ref 0 00002000:00100000:1.0:1789762963.437926:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0x11:0x0] (4194304->5242879) req@ffff981ebfdc9c00 x1876702015154688/t4294967440(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:H/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.437934:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc9c00 with x1876702015154688 ext(4194304->5242879) 00010000:00010000:1.0:1789762963.437941:0:14912:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 27 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.437961:0:14912:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 27 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762963.437970:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff981ebfdc9c00 00000100:00100000:2.0:1789762963.440141:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.440200:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015154816 00000100:00100000:1.0:1789762963.440209:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015154816 00010000:00010000:1.0:1789762963.440242:0:6664:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 27 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.440287:0:6664:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 66 00000100:00100000:1.0:1789762963.440300:0:6664:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2365:x1876702015154816:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762963.440318:0:6664:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1789762963.440334:0:6664:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb size from disk: 1048576 -> 5242880 00010000:00010000:1.0:1789762963.440342:0:6664:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 9 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x60000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 27 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.440355:0:6664:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 9 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x50000400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 27 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.440377:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005b162c57/0x416193e012b05c82 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 9 type: EXT [4194304->6291455] (req 4194304->5242879) gid 0 flags: 0x44801400020020 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8ee expref: 27 pid: 6649 timeout: 586 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.440394:0:6664:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762963.440398:0:6664:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(7 0) 00010000:00010000:1.0:1789762963.440406:0:6664:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c90 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8fc expref: 26 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.440417:0:6664:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c90 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40080000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8fc expref: 26 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.440453:0:6664:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c90 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40000000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8fc expref: 26 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.440468:0:6664:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb1c700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:6664:1876702022201472:192.168.203.48@tcp:105: 00000100:00100000:1.0:1789762963.440546:0:6664:0:(client.c:2715:ptlrpc_set_wait()) set ffff981e878baa80 going to sleep for 11 seconds 00000100:00100000:0.0:1789762963.446930:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762963.446954:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762963.446979:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.446991:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015154944 00000100:00100000:2.0:1789762963.446998:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015154944 00000100:00100000:1.0:1789762963.447004:0:6664:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb1c700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:6664:1876702022201472:192.168.203.48@tcp:105: 00000100:00100000:3.0:1789762963.447007:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015155072 00000100:00100000:3.0:1789762963.447016:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015155072 00000100:00100000:0.0:1789762963.447018:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015155200 00010000:00010000:1.0:1789762963.447023:0:6664:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762963.447024:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015155200 00002000:00100000:2.0:1789762963.447030:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x1a:0x0] (5242880->6291455) req@ffff981ebfdbb480 x1876702015154944/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762963.447047:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1a:0x0] (2097152->3145727) req@ffff981ebfdb9c00 x1876702015155072/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762963.447049:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbb480 with x1876702015154944 ext(5242880->6291455) 00002000:00100000:0.0:1789762963.447052:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x1a:0x0] (7340032->8388607) req@ffff981ebfdb9f80 x1876702015155200/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00010000:2.0:1789762963.447058:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdbb480 00000100:00100000:1.0:1789762963.447062:0:6664:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:2365:x1876702015154816:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 6763us (6920us total) trans 0 rc 0/0 00002000:00100000:3.0:1789762963.447082:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb9c00 with x1876702015155072 ext(2097152->3145727) 00000100:00100000:1.0:1789762963.447120:0:6664:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 66 00002000:00010000:3.0:1789762963.447130:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdb9c00 00002000:00100000:0.0:1789762963.447132:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb9f80 with x1876702015155200 ext(7340032->8388607) 00000100:00100000:2.0:1789762963.447137:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486681259506 time 486731187382 state 0x0 ref 1 00002000:00010000:0.0:1789762963.447139:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdb9f80 00000100:00100000:3.0:1789762963.447165:0:11723:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486681259506 time 486731187382 state 0x0 ref 2 00000100:00100000:2.0:1789762963.447181:0:12582:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486681259506 time 486731187382 state 0x0 ref 3 00000100:00100000:0.0:1789762963.447197:0:6654:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486681259506 time 486731187382 state 0x0 ref 3 00000100:00100000:3.0:1789762963.447242:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486731287382 time 486731187382 state 0x1 ref 3 00000100:00100000:2.0:1789762963.447252:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486731287382 time 486731187382 state 0x1 ref 3 00000100:00100000:0.0:1789762963.447263:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486731287382 time 486731187382 state 0x1 ref 3 00000100:00100000:3.0:1789762963.447313:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486751281846 time 486751181846 state 0x1 ref 3 00000100:00100000:3.0:1789762963.447328:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:2364:x1876702015155072:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762963.447329:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 486751332974 time 486751232974 state 0x1 ref 3 00000100:00100000:0.0:1789762963.447347:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 0 deadline 486751332974 time 486751252737 state 0x0 ref 3 00000100:00100000:2.0:1789762963.447347:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:2363:x1876702015154944:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762963.447395:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:2364:x1876702015155200:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762963.450083:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486803988982 time 486753988982 state 0x1 ref 7 00000100:00100000:2.0:1789762963.450114:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2366:x1876702015151616:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762963.450828:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.450873:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015155328 00000100:00100000:2.0:1789762963.450878:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015155328 00002000:00100000:2.0:1789762963.450898:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_006: refresh rw locks for [0x240000400:0x1a:0x0] (4194304->5242879) req@ffff981ebfb1c700 x1876702015155328/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762963.450914:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1c700 with x1876702015155328 ext(4194304->5242879) 00002000:00010000:2.0:1789762963.450919:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1c700 00000100:00100000:2.0:1789762963.450930:0:14912:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 0 deadline 486751332974 time 486751252737 state 0x0 ref 4 00000100:00100000:2.0:1789762963.450957:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 0 deadline 486751352737 time 486751252737 state 0x1 ref 4 00000100:00100000:1.0:1789762963.450987:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486751352737 time 486754891904 state 0x0 ref 4 00000100:00100000:1.0:1789762963.451001:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:2364:x1876702015155328:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762963.452227:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762963.452268:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015155456 00000100:00100000:0.0:1789762963.452273:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015155456 00000100:00100000:2.0:1789762963.452278:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00002000:00100000:0.0:1789762963.452296:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x11:0x0] (6291456->7340031) req@ffff981d81394a80 x1876702015155456/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762963.452305:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81394a80 with x1876702015155456 ext(6291456->7340031) 00002000:00010000:0.0:1789762963.452309:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81394a80 00000100:00100000:0.0:1789762963.452316:0:12581:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486751352737 time 486754891904 state 0x0 ref 5 00000100:00100000:0.0:1789762963.452347:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486754991904 time 486754891904 state 0x1 ref 5 00000100:00100000:0.0:1789762963.452365:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015155584 00000100:00100000:0.0:1789762963.452367:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015155584 00002000:00100000:0.0:1789762963.452382:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x11:0x0] (3145728->4194303) req@ffff981d81395c00 x1876702015155584/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789762963.452395:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395c00 with x1876702015155584 ext(3145728->4194303) 00002000:00010000:0.0:1789762963.452401:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81395c00 00000100:00100000:2.0:1789762963.452403:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486754991904 time 486756296219 state 0x0 ref 5 00000100:00100000:0.0:1789762963.452416:0:12581:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486754991904 time 486756296219 state 0x0 ref 6 00000100:00100000:2.0:1789762963.452424:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2363:x1876702015155456:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762963.452435:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486756396219 time 486756296219 state 0x1 ref 6 00000100:00100000:0.0:1789762963.452449:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 486756396219 time 486756358993 state 0x0 ref 6 00000100:00100000:0.0:1789762963.452458:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:2363:x1876702015155584:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762963.452554:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.452591:0:14913:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 9 min 3 max 116 00000100:00100000:2.0:1789762963.452599:0:14913:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_009' 00000100:00100000:1.0:1789762963.452752:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015155712 00000100:00100000:1.0:1789762963.452759:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015155712 00002000:00100000:1.0:1789762963.452780:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0x11:0x0] (7340032->8388607) req@ffff981ebfdba680 x1876702015155712/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.452788:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdba680 with x1876702015155712 ext(7340032->8388607) 00002000:00010000:1.0:1789762963.452791:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdba680 00000100:00100000:1.0:1789762963.452799:0:14913:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 486756396219 time 486756358993 state 0x0 ref 7 00000100:00100000:1.0:1789762963.452812:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 486756458993 time 486756358993 state 0x1 ref 7 00000100:00100000:2.0:1789762963.453212:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486756458993 time 486757118505 state 0x0 ref 7 00000100:00100000:2.0:1789762963.453221:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:2363:x1876702015155712:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762963.454464:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.454521:0:14913:0:(service.c:3462:ptlrpc_start_thread()) ost_io[0] started 10 min 3 max 116 00000100:00100000:3.0:1789762963.454528:0:14913:0:(service.c:3521:ptlrpc_start_thread()) starting thread 'll_ost_io00_010' 00000100:00100000:3.0:1789762963.454637:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015155840 00000100:00100000:3.0:1789762963.454642:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015155840 00002000:00100000:3.0:1789762963.454668:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0x11:0x0] (2097152->3145727) req@ffff981ebfb1f480 x1876702015155840/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762963.454679:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1f480 with x1876702015155840 ext(2097152->3145727) 00002000:00010000:3.0:1789762963.454684:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfb1f480 00000100:00100000:3.0:1789762963.454695:0:14913:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486756458993 time 486757118505 state 0x0 ref 8 00000100:00100000:3.0:1789762963.454709:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486757218505 time 486757118505 state 0x1 ref 8 00000100:00100000:0.0:1789762963.455190:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486757218505 time 486759094426 state 0x0 ref 8 00000100:00100000:0.0:1789762963.455205:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb1f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:2366:x1876702015155840:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:3.0:1789762963.455505:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2364:x1876702015155072:12345-192.168.203.48@tcp:4:dd.0 Request processed in 8179us (8546us total) trans 4294968259 rc 0/0 00000100:00100000:3.0:1789762963.455517:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1375 00002000:00100000:3.0:1789762963.455527:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1a:0x0] (2097152->3145727) req@ffff981ebfdb9c00 x1876702015155072/t4294968259(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762963.455537:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb9c00 with x1876702015155072 ext(2097152->3145727) 00002000:00010000:3.0:1789762963.455542:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdb9c00 00000100:00100000:1.0:1789762963.455627:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2364:x1876702015155200:12345-192.168.203.48@tcp:4:dd.0 Request processed in 8262us (8647us total) trans 4294968260 rc 0/0 00000100:00100000:1.0:1789762963.455641:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1377 00002000:00100000:1.0:1789762963.455652:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x1a:0x0] (7340032->8388607) req@ffff981ebfdb9f80 x1876702015155200/t4294968260(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.455662:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdb9f80 with x1876702015155200 ext(7340032->8388607) 00002000:00010000:1.0:1789762963.455666:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdb9f80 00000100:00100000:0.0:1789762963.460187:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.460215:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015155968 00000100:00100000:1.0:1789762963.460222:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015155968 00002000:00100000:1.0:1789762963.460246:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x1a:0x0] (3145728->4194303) req@ffff981ebfdbbb80 x1876702015155968/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.460254:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbbb80 with x1876702015155968 ext(3145728->4194303) 00002000:00010000:1.0:1789762963.460257:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdbbb80 00000100:00100000:1.0:1789762963.460267:0:14913:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486757218505 time 486759094426 state 0x0 ref 7 00000100:00100000:1.0:1789762963.460300:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486759194426 time 486759094426 state 0x1 ref 7 00000100:00100000:1.0:1789762963.460321:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486759194426 time 486764230891 state 0x0 ref 7 00000100:00100000:1.0:1789762963.460331:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:2364:x1876702015155968:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:2.0:1789762963.461397:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2363:x1876702015155712:12345-192.168.203.48@tcp:4:dd.0 Request processed in 8173us (8839us total) trans 4294967441 rc 0/0 00000100:00100000:2.0:1789762963.461408:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1381 00002000:00100000:2.0:1789762963.461415:0:17497:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x280000400:0x11:0x0] (7340032->8388607) req@ffff981ebfdba680 x1876702015155712/t4294967441(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762963.461423:0:17497:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdba680 with x1876702015155712 ext(7340032->8388607) 00002000:00010000:2.0:1789762963.461427:0:17497:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdba680 00000100:00100000:3.0:1789762963.462948:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2363:x1876702015155456:12345-192.168.203.48@tcp:4:dd.0 Request processed in 10530us (10722us total) trans 4294967442 rc 0/0 00000100:00100000:3.0:1789762963.462954:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1379 00002000:00100000:3.0:1789762963.462996:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x11:0x0] (6291456->7340031) req@ffff981d81394a80 x1876702015155456/t4294967442(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789762963.463004:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81394a80 with x1876702015155456 ext(6291456->7340031) 00002000:00010000:3.0:1789762963.463008:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81394a80 00000100:00100000:1.0:1789762963.463812:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2363:x1876702015155584:12345-192.168.203.48@tcp:4:dd.0 Request processed in 11353us (11532us total) trans 4294967443 rc 0/0 00000100:00100000:1.0:1789762963.463821:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1380 00002000:00100000:1.0:1789762963.463828:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x11:0x0] (3145728->4194303) req@ffff981d81395c00 x1876702015155584/t4294967443(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.463836:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81395c00 with x1876702015155584 ext(3145728->4194303) 00002000:00010000:1.0:1789762963.463840:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81395c00 00000100:00100000:1.0:1789762963.465045:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2364:x1876702015155328:12345-192.168.203.48@tcp:4:dd.0 Request processed in 14045us (14216us total) trans 4294968261 rc 0/0 00000100:00100000:1.0:1789762963.465054:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1378 00002000:00100000:1.0:1789762963.465062:0:14914:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_008: refresh rw locks for [0x240000400:0x1a:0x0] (4194304->5242879) req@ffff981ebfb1c700 x1876702015155328/t4294968261(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.465070:0:14914:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1c700 with x1876702015155328 ext(4194304->5242879) 00002000:00010000:1.0:1789762963.465074:0:14914:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfb1c700 00000100:00100000:1.0:1789762963.467999:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2363:x1876702015154944:12345-192.168.203.48@tcp:4:dd.0 Request processed in 20652us (21068us total) trans 4294968262 rc 0/0 00000100:00100000:1.0:1789762963.468010:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1376 00002000:00100000:1.0:1789762963.468020:0:12582:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_005: refresh rw locks for [0x240000400:0x1a:0x0] (5242880->6291455) req@ffff981ebfdbb480 x1876702015154944/t4294968262(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.468031:0:12582:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbb480 with x1876702015154944 ext(5242880->6291455) 00002000:00010000:1.0:1789762963.468037:0:12582:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdbb480 00000100:00100000:1.0:1789762963.469167:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb1f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2366:x1876702015155840:12345-192.168.203.48@tcp:4:dd.0 Request processed in 13962us (14699us total) trans 4294967444 rc 0/0 00000100:00100000:1.0:1789762963.469178:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1382 00002000:00100000:1.0:1789762963.469187:0:17498:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_010: refresh rw locks for [0x280000400:0x11:0x0] (2097152->3145727) req@ffff981ebfb1f480 x1876702015155840/t4294967444(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.469198:0:17498:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfb1f480 with x1876702015155840 ext(2097152->3145727) 00002000:00010000:1.0:1789762963.469204:0:17498:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfb1f480 00000100:00100000:0.0:1789762963.469384:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2366:x1876702015151616:12345-192.168.203.48@tcp:4:dd.500 Request processed in 19268us (227572us total) trans 4294968263 rc 0/0 00000100:00100000:0.0:1789762963.469392:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1366 00002000:00100000:0.0:1789762963.469400:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x19:0x0] (5242880->6291455) req@ffff981d81ea7b80 x1876702015151616/t4294968263(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:0.0:1789762963.469408:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81ea7b80 with x1876702015151616 ext(5242880->6291455) 00002000:00010000:0.0:1789762963.469412:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81ea7b80 00000100:00100000:1.0:1789762963.470053:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2364:x1876702015155968:12345-192.168.203.48@tcp:4:dd.0 Request processed in 9722us (9867us total) trans 4294968264 rc 0/0 00000100:00100000:1.0:1789762963.470062:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1383 00000100:00100000:1.0:1789762963.470071:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486759194426 time 486764230891 state 0x2 ref 0 00002000:00100000:1.0:1789762963.470080:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_007: refresh rw locks for [0x240000400:0x1a:0x0] (3145728->4194303) req@ffff981ebfdbbb80 x1876702015155968/t4294968264(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.470117:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdbbb80 with x1876702015155968 ext(3145728->4194303) 00002000:00010000:1.0:1789762963.470121:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdbbb80 00000100:00100000:2.0:1789762963.470605:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.470625:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.470651:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015156096 00000100:00100000:1.0:1789762963.470658:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015156096 00000100:00100000:0.0:1789762963.470666:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015156224 00000100:00100000:0.0:1789762963.470672:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015156224 00002000:00100000:1.0:1789762963.470685:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x1a:0x0] (6291456->7340031) req@ffff981d81397b80 x1876702015156096/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.470693:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81397b80 with x1876702015156096 ext(6291456->7340031) 00002000:00100000:0.0:1789762963.470694:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0x11:0x0] (1048576->2097151) req@ffff981d81394380 x1876702015156224/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00010000:1.0:1789762963.470697:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81397b80 00002000:00100000:0.0:1789762963.470706:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81394380 with x1876702015156224 ext(1048576->2097151) 00000100:00100000:1.0:1789762963.470714:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486759194426 time 486764230891 state 0x0 ref 1 00002000:00010000:0.0:1789762963.470715:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81394380 00000100:00100000:1.0:1789762963.470724:0:6654:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486759194426 time 486764230891 state 0x0 ref 2 00000100:00100000:0.0:1789762963.470738:0:14912:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486759194426 time 486764230891 state 0x0 ref 2 00000100:00100000:1.0:1789762963.470769:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486764330891 time 486764230891 state 0x1 ref 2 00000100:00100000:0.0:1789762963.470778:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486764330891 time 486764230891 state 0x1 ref 2 00000100:00100000:2.0:1789762963.470797:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486774801914 time 486774701914 state 0x1 ref 2 00000100:00100000:2.0:1789762963.470817:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81397b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2365:x1876702015156096:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762963.470826:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 486774801914 time 486774724752 state 0x0 ref 2 00000100:00100000:1.0:1789762963.470840:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:2366:x1876702015156224:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:0.0:1789762963.473018:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.473058:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015156480 00000100:00100000:1.0:1789762963.473064:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015156480 00000100:00100000:1.0:1789762963.473122:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 121 00000100:00100000:1.0:1789762963.473133:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+17:15811:x1876702015156480:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:2.0:1789762963.473167:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81397b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2365:x1876702015156096:12345-192.168.203.48@tcp:4:dd.0 Request processed in 2351us (2562us total) trans 4294968265 rc 0/0 00000100:00100000:2.0:1789762963.473174:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1384 00002000:00100000:2.0:1789762963.473180:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1a:0x0] (6291456->7340031) req@ffff981d81397b80 x1876702015156096/t4294968265(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762963.473188:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81397b80 with x1876702015156096 ext(6291456->7340031) 00002000:00010000:2.0:1789762963.473191:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981d81397b80 00000100:00100000:1.0:1789762963.473356:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15811:x1876702015156480:12345-192.168.203.48@tcp:35:dd.0 Request processed in 226us (340us total) trans 4294967624 rc 0/0 00000100:00100000:1.0:1789762963.473363:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 121 00000100:00100000:0.0:1789762963.474458:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.474511:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015156608 00000100:00100000:1.0:1789762963.474517:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015156608 00002000:00100000:1.0:1789762963.474547:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x11:0x0] (5242880->6291455) req@ffff981ebc555f80 x1876702015156608/t0(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.474557:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc555f80 with x1876702015156608 ext(5242880->6291455) 00002000:00010000:1.0:1789762963.474561:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc555f80 00000100:00100000:1.0:1789762963.474572:0:12581:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 486774801914 time 486774724752 state 0x0 ref 2 00000100:00100000:1.0:1789762963.474611:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 1 deadline 486774824752 time 486774724752 state 0x1 ref 2 00000100:00100000:1.0:1789762963.474636:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486774824752 time 486778544242 state 0x0 ref 2 00000100:00100000:1.0:1789762963.474649:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2366:x1876702015156608:12345-192.168.203.48@tcp:4:dd.0 00000100:00100000:1.0:1789762963.478032:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2366:x1876702015156608:12345-192.168.203.48@tcp:4:dd.0 Request processed in 3382us (3570us total) trans 4294967445 rc 0/0 00000100:00100000:1.0:1789762963.478041:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1386 00002000:00100000:1.0:1789762963.478050:0:12581:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x11:0x0] (5242880->6291455) req@ffff981ebc555f80 x1876702015156608/t4294967445(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789762963.478061:0:12581:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebc555f80 with x1876702015156608 ext(5242880->6291455) 00002000:00010000:1.0:1789762963.478065:0:12581:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebc555f80 00000100:00100000:2.0:1789762963.479249:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2366:x1876702015156224:12345-192.168.203.48@tcp:4:dd.0 Request processed in 8412us (8620us total) trans 4294967446 rc 0/0 00000100:00100000:2.0:1789762963.479257:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1385 00000100:00100000:2.0:1789762963.479265:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981e87842e00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 486774824752 time 486778544242 state 0x2 ref 0 00002000:00100000:2.0:1789762963.479273:0:6654:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x11:0x0] (1048576->2097151) req@ffff981d81394380 x1876702015156224/t4294967446(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762963.479280:0:6654:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981d81394380 with x1876702015156224 ext(1048576->2097151) 00002000:00010000:2.0:1789762963.479284:0:6654:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981d81394380 00000100:00100000:1.0:1789762963.500164:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486854053083 time 486804053083 state 0x1 ref 6 00000100:00100000:1.0:1789762963.500186:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+36:2365:x1876702015151744:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762963.502575:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702015151744:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2393us (254485us total) trans 4294968266 rc 0/0 00000100:00100000:2.0:1789762963.502584:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1367 00002000:00100000:2.0:1789762963.502592:0:17498:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_010: refresh rw locks for [0x240000400:0x19:0x0] (6291456->7340031) req@ffff981ebfdc8380 x1876702015151744/t4294968266(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762963.502611:0:17498:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc8380 with x1876702015151744 ext(6291456->7340031) 00002000:00010000:2.0:1789762963.502616:0:17498:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdc8380 00000100:00100000:0.0:1789762963.550266:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486904159724 time 486854159724 state 0x1 ref 5 00000100:00100000:0.0:1789762963.550291:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2366:x1876702015151872:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:2.0:1789762963.552888:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2366:x1876702015151872:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2603us (298214us total) trans 4294968267 rc 0/0 00000100:00100000:2.0:1789762963.552896:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1368 00002000:00100000:2.0:1789762963.552906:0:6652:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x19:0x0] (7340032->8388607) req@ffff981ebfdca680 x1876702015151872/t4294968267(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:2.0:1789762963.552914:0:6652:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdca680 with x1876702015151872 ext(7340032->8388607) 00002000:00010000:2.0:1789762963.552919:0:6652:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebfdca680 00000100:00100000:1.0:1789762963.600340:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 486954238726 time 486904238726 state 0x1 ref 4 00000100:00100000:1.0:1789762963.600362:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:2366:x1876702015152000:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762963.602876:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb19f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2366:x1876702015152000:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2521us (341804us total) trans 4294967447 rc 0/0 00000100:00100000:1.0:1789762963.602884:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1369 00002000:00100000:1.0:1789762963.602895:0:14913:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_007: refresh rw locks for [0x280000400:0x10:0x0] (4194304->5242879) req@ffff981eadb19f80 x1876702015152000/t4294967447(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762963.602904:0:14913:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eadb19f80 with x1876702015152000 ext(4194304->5242879) 00002000:00010000:1.0:1789762963.602909:0:14913:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eadb19f80 00000100:00100000:0.0:1789762963.650409:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487004305947 time 486954305947 state 0x1 ref 3 00000100:00100000:0.0:1789762963.650428:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702015152128:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:1.0:1789762963.653061:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2365:x1876702015152128:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2637us (385692us total) trans 4294967448 rc 0/0 00000100:00100000:1.0:1789762963.653071:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1370 00002000:00100000:1.0:1789762963.653080:0:14912:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_006: refresh rw locks for [0x280000400:0x10:0x0] (5242880->6291455) req@ffff981eadb1a680 x1876702015152128/t4294967448(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:1.0:1789762963.653113:0:14912:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eadb1a680 with x1876702015152128 ext(5242880->6291455) 00002000:00010000:1.0:1789762963.653119:0:14912:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eadb1a680 00000100:00100000:3.0:1789762963.700503:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487054401664 time 487004401664 state 0x1 ref 2 00000100:00100000:3.0:1789762963.700521:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eadb1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:2365:x1876702015152256:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762963.706523:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eadb1ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+41:2365:x1876702015152256:12345-192.168.203.48@tcp:4:dd.500 Request processed in 6005us (401972us total) trans 4294967449 rc 0/0 00000100:00100000:3.0:1789762963.706531:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1371 00002000:00100000:3.0:1789762963.706541:0:6653:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x10:0x0] (6291456->7340031) req@ffff981eadb1ad80 x1876702015152256/t4294967449(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762963.706549:0:6653:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981eadb1ad80 with x1876702015152256 ext(6291456->7340031) 00002000:00010000:3.0:1789762963.706554:0:6653:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981eadb1ad80 00000100:00100000:3.0:1789762963.750560:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487054401664 time 487054456497 state 0x0 ref 1 00000100:00100000:3.0:1789762963.750579:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+40:2365:x1876702015152384:12345-192.168.203.48@tcp:4:dd.500 00000100:00100000:3.0:1789762963.753246:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:2365:x1876702015152384:12345-192.168.203.48@tcp:4:dd.500 Request processed in 2671us (441946us total) trans 4294967450 rc 0/0 00000100:00100000:3.0:1789762963.753254:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1372 00000100:00100000:3.0:1789762963.753262:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487054401664 time 487054456497 state 0x2 ref 0 00002000:00100000:3.0:1789762963.753275:0:17497:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_009: refresh rw locks for [0x280000400:0x10:0x0] (7340032->8388607) req@ffff981ebfdc8000 x1876702015152384/t4294967450(0) o4->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/448 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.500' uid:500 gid:500 projid:0 00002000:00100000:3.0:1789762963.753284:0:17497:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebfdc8000 with x1876702015152384 ext(7340032->8388607) 00002000:00010000:3.0:1789762963.753289:0:17497:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff981ebfdc8000 00000100:00100000:0.0:1789762963.787764:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762963.787794:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.787825:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015156736 00000100:00100000:2.0:1789762963.787834:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015156736 00000100:00100000:3.0:1789762963.787842:0:5792:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015156864 00000100:00100000:3.0:1789762963.787849:0:5792:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015156864 00000100:00100000:2.0:1789762963.787892:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 35 00000100:00100000:3.0:1789762963.787901:0:5792:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 36 00000100:00100000:2.0:1789762963.787907:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e86f00050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+39:5592:x1876702015156736:12345-192.168.203.48@tcp:103:ldlm_bl.0 00000100:00100000:3.0:1789762963.787917:0:5792:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc556680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+42:5593:x1876702015156864:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762963.787920:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1789762963.787927:0:5792:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 7 locks, starting at 0 00002000:00010000:2.0:1789762963.787937:0:5791:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:3.0:1789762963.787940:0:5792:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:2.0:1789762963.787941:0:5791:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb mtime from disk: 0 -> 1789762963 00002000:00010000:3.0:1789762963.787946:0:5792:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb mtime from disk: 0 -> 1789762963 00002000:00010000:2.0:1789762963.787947:0:5791:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb ctime from disk: 0 -> 1789762963 00002000:00010000:3.0:1789762963.787954:0:5792:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb ctime from disk: 0 -> 1789762963 00002000:00010000:3.0:1789762963.787958:0:5792:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:2.0:1789762963.787980:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d57ee577/0x416193e012b05c20 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8af expref: 39 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.787980:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002b972c92/0x416193e012b05c2e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8b6 expref: 42 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789762963.788023:0:5791:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb size from disk: 0 -> 8388608 00002000:00010000:3.0:1789762963.788024:0:5792:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x11:0x0] updating lvb size from disk: 5242880 -> 8388608 00002000:00010000:2.0:1789762963.788027:0:5791:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb mtime from disk: 0 -> 1789762963 00002000:00010000:2.0:1789762963.788030:0:5791:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb ctime from disk: 0 -> 1789762963 00010000:00010000:3.0:1789762963.788034:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c253e283/0x416193e012b05c90 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 8 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8fc expref: 41 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.788040:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05c74 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 1048576->2097151) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8e0 expref: 38 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.788052:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004887b6d3/0x416193e012b05c5f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 7 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8cb expref: 40 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.788080:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762963.788118:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c1889e1e/0x416193e012b05c66 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 6 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d2 expref: 39 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.788135:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ceb8603d/0x416193e012b05c7b lrc: 0/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 5 type: EXT [7340032->18446744073709551615] (req 7340032->8388607) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8e7 expref: 38 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.788162:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bc5cda3e/0x416193e012b05c6d lrc: 0/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8d9 expref: 37 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762963.788170:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e86f00050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:5592:x1876702015156736:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 267us (408us total) trans 0 rc 0/0 00010000:00010000:3.0:1789762963.788177:0:5792:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c462886a/0x416193e012b05c89 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44801000020000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8f5 expref: 36 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762963.788183:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 35 00010000:00010000:3.0:1789762963.788198:0:5792:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789762963.788225:0:5792:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc556680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:5593:x1876702015156864:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 312us (428us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762963.788234:0:5792:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 36 00000100:00100000:2.0:1789762963.827051:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.827200:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015157120 00000100:00100000:3.0:1789762963.827208:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015157120 00000100:00100000:3.0:1789762963.827255:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 379 00000100:00100000:3.0:1789762963.827264:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+18:15838:x1876702015157120:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762963.827284:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762963.827300:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b05c97 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd903 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762963.827301:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762963.827389:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015157376 00000100:00100000:1.0:1789762963.827395:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015157376 00000100:00100000:1.0:1789762963.827426:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 380 00010000:00010000:3.0:1789762963.827433:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05c9e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.827434:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+20:15839:x1876702015157376:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762963.827448:0:5806:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762963.827452:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05c9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.827459:0:5806:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05ca5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd90a expref: -99 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.827465:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05c9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762963.827477:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3f:0x0]: returning size 16777216 00010000:00010000:1.0:1789762963.827532:0:5806:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b05cb3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.827540:0:5806:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b05cb3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.827547:0:5806:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b05cb3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5806 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762963.827553:0:5806:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3f:0x0]: returning size 16777216 00010000:00010000:3.0:1789762963.827624:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b05c97 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3d:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd903 expref: 25 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.827629:0:5806:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfce00/0x416193e012b05ca5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3d:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd90a expref: 25 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.827656:0:5806:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b05cb3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd90a expref: 23 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.827664:0:5806:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3f:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:3.0:1789762963.827666:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05c9e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd903 expref: 23 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.827667:0:5806:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789762963.827672:0:5806:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87dfc000, rc 0) 00010000:00010000:3.0:1789762963.827680:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3f:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:3.0:1789762963.827682:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762963.827685:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880ab800, rc 0) 00000100:00100000:1.0:1789762963.827717:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15839:x1876702015157376:12345-192.168.203.48@tcp:101:dd.0 Request processed in 284us (420us total) trans 4294967626 rc 0/0 00000100:00100000:1.0:1789762963.827728:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 380 00000100:00100000:3.0:1789762963.827729:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15838:x1876702015157120:12345-192.168.203.48@tcp:101:dd.0 Request processed in 461us (682us total) trans 4294967625 rc 0/0 00000100:00100000:3.0:1789762963.827737:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 379 00000100:00100000:2.0:1789762963.829212:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.829247:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015157632 00000100:00100000:3.0:1789762963.829251:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015157632 00000100:00100000:3.0:1789762963.829277:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 381 00000100:00100000:3.0:1789762963.829282:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:15840:x1876702015157632:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:3.0:1789762963.829292:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762963.829300:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05cc1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd911 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.829334:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05cc8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.829341:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05cc8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.829347:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05cc8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762963.829356:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3f:0x0]: returning size 16777216 00010000:00010000:3.0:1789762963.829418:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05cc1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3d:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd911 expref: 25 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.829430:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05cc8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd911 expref: 24 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.829435:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3f:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:3.0:1789762963.829437:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762963.829439:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880ab000, rc 0) 00000100:00100000:3.0:1789762963.829467:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:15840:x1876702015157632:12345-192.168.203.48@tcp:101:dd.0 Request processed in 185us (256us total) trans 4294967627 rc 0/0 00000100:00100000:3.0:1789762963.829473:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 381 00000100:00100000:0.0:1789762963.829574:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762963.829601:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762963.829629:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.829656:0:8405:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015157760 00000100:00100000:2.0:1789762963.829663:0:8405:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015157760 00000100:00100000:1.0:1789762963.829669:0:6664:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015157888 00000100:00100000:1.0:1789762963.829676:0:6664:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015157888 00000100:00100000:3.0:1789762963.829693:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015158016 00000100:00100000:3.0:1789762963.829698:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015158016 00000100:00100000:2.0:1789762963.829702:0:8405:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 37 00000100:00100000:2.0:1789762963.829714:0:8405:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc557b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2363:x1876702015157760:12345-192.168.203.48@tcp:103:ldlm_bl.0 00000100:00100000:1.0:1789762963.829719:0:6664:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 38 00010000:00010000:2.0:1789762963.829725:0:8405:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:1.0:1789762963.829733:0:6664:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc556300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:2363:x1876702015157888:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762963.829742:0:8405:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87dfc000/0x416193e012b05cb3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd90a expref: 25 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.829743:0:6664:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:3.0:1789762963.829744:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 226 00000100:00100000:3.0:1789762963.829760:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc554a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:15838:x1876702015158016:12345-192.168.203.48@tcp:101:dd.0 00010000:00010000:1.0:1789762963.829761:0:6664:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05c9e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd903 expref: 24 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.829762:0:8405:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3f:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:2.0:1789762963.829770:0:8405:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762963.829772:0:6649:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1789762963.829781:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00010000:00010000:1.0:1789762963.829782:0:6664:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3f:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:1.0:1789762963.829787:0:6664:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762963.829790:0:6649:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000020801b7b/0x416193e012b05cd6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd918 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762963.829802:0:8405:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc557b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2363:x1876702015157760:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 91us (228us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762963.829808:0:8405:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 37 00000100:00100000:1.0:1789762963.829815:0:6664:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc556300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2363:x1876702015157888:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 85us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762963.829821:0:6664:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 38 00002000:00010000:3.0:1789762963.829852:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x19:0x0] initial LVB size: 8388608, mtime: 0x6aad9d93, atime: 0x0, ctime: 0x6aad9d93, blocks: 0x1 00000100:00100000:2.0:1789762963.829857:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015158272 00000100:00100000:2.0:1789762963.829860:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015158272 00000100:00100000:2.0:1789762963.829887:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 122 00000100:00100000:2.0:1789762963.829893:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:15839:x1876702015158272:12345-192.168.203.48@tcp:35:dd.0 00010000:00010000:3.0:1789762963.829922:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=487s v=(0 0) 00010000:00010000:3.0:1789762963.829927:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762963.829963:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000020801b7b/0x416193e012b05cd6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd918 expref: 38 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.829977:0:6649:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000020801b7b/0x416193e012b05cd6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd918 expref: 38 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.829989:0:6649:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aa600, rc 0) 00000100:00100000:2.0:1789762963.829993:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15839:x1876702015158272:12345-192.168.203.48@tcp:35:dd.0 Request processed in 99us (213us total) trans 4294967628 rc 0/0 00000100:00100000:2.0:1789762963.830012:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 122 00000100:00100000:3.0:1789762963.830024:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc554a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:15838:x1876702015158016:12345-192.168.203.48@tcp:101:dd.0 Request processed in 266us (391us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762963.830033:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 226 00000100:00100000:0.0:1789762963.831079:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.831158:0:5791:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015158400 00000100:00100000:2.0:1789762963.831164:0:5791:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015158400 00000100:00100000:2.0:1789762963.831199:0:5791:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 39 00000100:00100000:2.0:1789762963.831208:0:5791:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:2366:x1876702015158400:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789762963.831218:0:5791:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762963.831230:0:5791:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05cc8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd911 expref: 23 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.831243:0:5791:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3f:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:2.0:1789762963.831246:0:5791:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789762963.831276:0:5791:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:2366:x1876702015158400:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 70us (197us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762963.831285:0:5791:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 39 00000100:00100000:1.0:1789762963.831363:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.831425:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015158656 00000100:00100000:3.0:1789762963.831429:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015158656 00000100:00100000:3.0:1789762963.831455:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 123 00000100:00100000:3.0:1789762963.831461:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+22:15840:x1876702015158656:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:3.0:1789762963.831557:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15840:x1876702015158656:12345-192.168.203.48@tcp:35:dd.0 Request processed in 94us (195us total) trans 4294967629 rc 0/0 00000100:00100000:3.0:1789762963.831567:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 123 00000100:00100000:0.0:1789762963.834925:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.834968:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015158784 00000100:00100000:2.0:1789762963.834976:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015158784 00002000:00100000:2.0:1789762963.835004:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x19:0x0] (0->1048575) req@ffff981ebac6d180 x1876702015158784/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/0 e 0 to 0 dl 1789762974 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762963.835015:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac6d180 with x1876702015158784 ext(0->1048575) 00002000:00010000:2.0:1789762963.835020:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac6d180 00000100:00100000:2.0:1789762963.835036:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 432764443504 time 432764382211 state 0x0 ref 1 00000100:00100000:2.0:1789762963.835045:0:11723:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x2, head sequence 11: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 432764443504 time 432764382211 state 0x0 ref 1 00000100:00100000:2.0:1789762963.835059:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 432764482211 time 432764382211 state 0x1 ref 1 00000100:00100000:2.0:1789762963.835116:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 432764482211 time 487139022745 state 0x0 ref 1 00000100:00100000:2.0:1789762963.835128:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702015158784:12345-192.168.203.48@tcp:3:dd.0 00000100:00100000:2.0:1789762963.840741:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2365:x1876702015158784:12345-192.168.203.48@tcp:3:dd.0 Request processed in 5604us (5806us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762963.840755:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1387 00000100:00100000:2.0:1789762963.840765:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebac66c00 key jobid={dd.0}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={0}&gid={0}&projid={0}&nodemap={-1} rule@0xffff981e87804000 gen 0 rate 10000 depth 3 token 2 deadline 432764482211 time 487139022745 state 0x2 ref 0 00002000:00100000:2.0:1789762963.840780:0:11723:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x19:0x0] (0->1048575) req@ffff981ebac6d180 x1876702015158784/t0(0) o3->f1bd28de-ee78-4d18-bad3-fa7cec968893@192.168.203.48@tcp:744/0 lens 488/440 e 0 to 0 dl 1789762974 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789762963.840789:0:11723:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff981ebac6d180 with x1876702015158784 ext(0->1048575) 00002000:00010000:2.0:1789762963.840794:0:11723:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff981ebac6d180 00000100:00100000:0.0:1789762963.877182:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.877235:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015159040 00000100:00100000:2.0:1789762963.877245:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015159040 00000100:00100000:2.0:1789762963.877292:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 124 00000100:00100000:2.0:1789762963.877301:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7ad6050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+23:15852:x1876702015159040:12345-192.168.203.48@tcp:35:dd.0 00000100:00100000:2.0:1789762963.877485:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7ad6050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:15852:x1876702015159040:12345-192.168.203.48@tcp:35:dd.0 Request processed in 185us (311us total) trans 4294967630 rc 0/0 00000100:00100000:2.0:1789762963.877491:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 124 00000100:00100000:1.0:1789762963.887193:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.887244:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015159168 00000100:00100000:2.0:1789762963.887251:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015159168 00000100:00100000:2.0:1789762963.887287:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 382 00000100:00100000:2.0:1789762963.887297:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+24:15854:x1876702015159168:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:2.0:1789762963.887325:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762963.887340:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05cdd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd91f expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.887384:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93c00/0x416193e012b05ce4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.887395:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93c00/0x416193e012b05ce4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.887404:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93c00/0x416193e012b05ce4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762963.887424:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3d:0x0]: returning size 16384 00000004:00200000:2.0:1789762963.887427:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762963.887437:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93c00/0x416193e012b05ce4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.887453:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05cdd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd91f expref: 26 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.887464:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93c00/0x416193e012b05ce4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd91f expref: 25 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.887470:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e9272c900) 00010000:00010000:2.0:1789762963.887472:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762963.887475:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfc93c00, rc 0) 00000100:00100000:2.0:1789762963.887501:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:15854:x1876702015159168:12345-192.168.203.48@tcp:101:rm.0 Request processed in 207us (312us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762963.887513:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 382 00000100:00100000:1.0:1789762963.888567:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.888607:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015159424 00000100:00100000:3.0:1789762963.888612:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015159424 00000100:00100000:3.0:1789762963.888641:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 383 00000100:00100000:3.0:1789762963.888648:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc49ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+25:15854:x1876702015159424:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:3.0:1789762963.888655:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762963.888663:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ceb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd926 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762963.888703:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3d:0x0]: returning size 16384 00010000:00010000:3.0:1789762963.888753:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ceb lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd926 expref: 27 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.888763:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:3.0:1789762963.888765:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789762963.888768:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aa400/0x416193e012b05ceb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd926 expref: 27 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.888775:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aa400, rc 0) 00000100:00100000:3.0:1789762963.888802:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc49ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:15854:x1876702015159424:12345-192.168.203.48@tcp:101:rm.0 Request processed in 156us (237us total) trans 4294967631 rc 301/301 00000100:00100000:3.0:1789762963.888807:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 383 00000100:00100000:0.0:1789762963.890300:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.890334:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015159552 00000100:00100000:3.0:1789762963.890340:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015159552 00000100:00100000:3.0:1789762963.890366:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 125 00000100:00100000:3.0:1789762963.890374:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:15854:x1876702015159552:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:3.0:1789762963.891992:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:15854:x1876702015159552:12345-192.168.203.48@tcp:37:rm.0 Request processed in 1619us (1692us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762963.891998:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 125 00000100:00100000:1.0:1789762963.893224:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.893257:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015159808 00000100:00100000:2.0:1789762963.893263:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015159808 00000100:00100000:2.0:1789762963.893289:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 126 00000100:00100000:2.0:1789762963.893294:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+26:15854:x1876702015159808:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:2.0:1789762963.893373:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:15854:x1876702015159808:12345-192.168.203.48@tcp:35:rm.0 Request processed in 78us (151us total) trans 4294967632 rc 0/0 00000100:00100000:2.0:1789762963.893378:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 126 00000100:00100000:0.0:1789762963.894052:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.894081:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015160064 00000100:00100000:3.0:1789762963.894103:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015160064 00000100:00100000:3.0:1789762963.894133:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 384 00000100:00100000:3.0:1789762963.894140:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:15854:x1876702015160064:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:3.0:1789762963.894162:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762963.894171:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b05cf9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd92d expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.894208:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05d00 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.894215:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05d00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3d:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.894221:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05d00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3d:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762963.894227:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3d:0x0]: returning size 16384 00010000:00010000:3.0:1789762963.894272:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aae00/0x416193e012b05cf9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd92d expref: 30 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.894288:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05d00 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3d:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd92d expref: 29 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.894303:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e9272c900) 00010000:00010000:3.0:1789762963.894307:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:3.0:1789762963.894309:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aac00, rc 0) 00000100:00100000:3.0:1789762963.894339:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15854:x1876702015160064:12345-192.168.203.48@tcp:101:rm.0 Request processed in 200us (288us total) trans 4294967633 rc 0/0 00000100:00100000:3.0:1789762963.894345:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 384 00000100:00100000:1.0:1789762963.895073:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.895123:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015160192 00000100:00100000:2.0:1789762963.895129:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015160192 00000100:00100000:2.0:1789762963.895170:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 385 00000100:00100000:2.0:1789762963.895179:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81ea6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15854:x1876702015160192:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:2.0:1789762963.895212:0:8165:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7719616512 avail 7715422208 00000100:00100000:2.0:1789762963.895248:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81ea6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15854:x1876702015160192:12345-192.168.203.48@tcp:41:rm.0 Request processed in 71us (176us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762963.895255:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 385 00000100:00100000:1.0:1789762963.895963:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.895984:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015160448 00000100:00100000:3.0:1789762963.895989:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015160448 00000100:00100000:3.0:1789762963.896012:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 386 00000100:00100000:3.0:1789762963.896018:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4970a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15854:x1876702015160448:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:3.0:1789762963.896038:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1789762963.896048:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b05c4a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8bd expref: 29 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896058:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762963.896066:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93c00/0x416193e012b05ce4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd91f expref: 28 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896073:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e9272c900) 00010000:00010000:3.0:1789762963.896075:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762963.896083:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05d0e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896105:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05d0e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896114:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05d0e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896122:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05d15 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896127:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05d15 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896131:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05d15 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896198:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05d1c lrc: 3/0,1 mode: --/EX res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896204:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05d1c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x40:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896208:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05d1c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x40:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762963.896678:0:6611:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].16, 64 off9216 00000004:00080000:3.0:1789762963.896686:0:6611:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 487200598334ns, next at 488200596758ns, rc = 0 00000040:00080000:3.0:1789762963.896717:0:6611:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].25, 64 off9792 00000004:00080000:3.0:1789762963.896721:0:6611:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 487200634118ns, next at 488200633589ns, rc = 0 00010000:00010000:3.0:1789762963.896757:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05d23 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896764:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05d23 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896773:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05d23 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896779:0:6611:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05d23 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896784:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05d23 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896788:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05d23 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896803:0:6611:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762963.896812:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05d1c lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x40:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896816:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05d1c lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x40:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896821:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05d1c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x40:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896826:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762963.896829:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762963.896830:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05d1c lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x40:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.896836:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05d23 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896838:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05d1c lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x40:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.896855:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762963.896858:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:3.0:1789762963.896861:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789762963.896865:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05d1c lrc: 2/0,0 mode: --/EX res: [0x200000401:0x40:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.896866:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05d23 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896875:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab000/0x416193e012b05d1c lrc: 0/0,0 mode: --/EX res: [0x200000401:0x40:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.896880:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05d23 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896888:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05d0e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.896892:0:6663:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff981e9272cf00) 00010000:00010000:3.0:1789762963.896894:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05d0e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.896895:0:6663:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789762963.896902:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05d23 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x40:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896903:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05d0e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896918:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aaa00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762963.896921:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b05d23 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x40:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896922:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05d0e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896935:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05d0e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.896944:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e9272c900) 00010000:00010000:3.0:1789762963.896946:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762963.896948:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05d0e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.896951:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762963.896953:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aaa00/0x416193e012b05d0e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762963.896954:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762963.896958:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00010000:3.0:1789762963.896964:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000df5b8b65 reply state 000000004aeaac3e transno 4294967634 ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05d15 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762963.896975:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762963.897007:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4970a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15854:x1876702015160448:12345-192.168.203.48@tcp:36:rm.0 Request processed in 989us (1046us total) trans 4294967634 rc 0/0 00000100:00100000:3.0:1789762963.897013:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 386 00010000:00010000:3.0:1789762963.897465:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05d15 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.897471:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05d15 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.897475:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05d15 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.897478:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aba00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762963.897480:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05d15 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.897485:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05d15 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.897491:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x992ad74f (ffff981e9272cc00) 00010000:00010000:3.0:1789762963.897493:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762963.897495:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05d15 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.897499:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05d15 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x992ad74f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.897916:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.897936:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015160704 00000100:00100000:3.0:1789762963.897975:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015160704 00000100:00100000:3.0:1789762963.897999:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 387 00000100:00100000:3.0:1789762963.898004:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4972d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:15854:x1876702015160704:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:3.0:1789762963.898023:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789762963.898033:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05c12 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8a8 expref: 28 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898042:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3f:0x0].0x0 (ffff981ea0aa3200) 00010000:00010000:3.0:1789762963.898048:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762963.898055:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d2a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898061:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d2a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898066:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d2a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898073:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05d31 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898077:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05d31 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898081:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05d31 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898137:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d38 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x3f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898143:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d38 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898167:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d38 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789762963.898404:0:5805:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].26, 64 off9856 00000040:00080000:3.0:1789762963.898415:0:5805:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].17, 64 off9280 00010000:00010000:3.0:1789762963.898442:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05d3f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898449:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05d3f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898454:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05d3f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898458:0:5805:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05d3f lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898462:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05d3f lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898467:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05d3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898479:0:5805:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762963.898486:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d38 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898491:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d38 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.898493:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762963.898496:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d38 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.898499:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05d3f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3f:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898519:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec05200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762963.898520:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec05400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762963.898525:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d38 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.898526:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05d3f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3f:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898542:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d38 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x3f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.898543:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05d3f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3f:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40018480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898559:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3f:0x0].0x0 (ffff981e9272c600) 00010000:00010000:3.0:1789762963.898562:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789762963.898566:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d38 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x3f:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898578:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d38 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x3f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.898584:0:5794:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3f:0x0].0x0 (ffff981e9272c600) 00010000:00010000:3.0:1789762963.898588:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d2a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.898589:0:5794:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789762963.898598:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05d3f lrc: 1/0,0 mode: --/PW res: [0x200000401:0x3f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898598:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d2a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898621:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d2a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.898622:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05d3f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898636:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec04e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762963.898640:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d2a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898645:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d2a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.898647:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762963.898650:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762963.898653:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789762963.898658:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e9272c900) 00010000:00010000:3.0:1789762963.898662:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762963.898664:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d2a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.898668:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d2a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762963.898675:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000facfbc32 reply state 000000000f071943 transno 4294967635 ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05d31 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762963.898679:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762963.898709:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4972d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15854:x1876702015160704:12345-192.168.203.48@tcp:36:rm.0 Request processed in 704us (795us total) trans 4294967635 rc 0/0 00000100:00100000:3.0:1789762963.898714:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 387 00010000:00010000:1.0:1789762963.899188:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05d31 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.899199:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05d31 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.899206:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05d31 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.899213:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec04a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762963.899217:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05d31 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.899225:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05d31 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.899235:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x77289514 (ffff981e9272ce00) 00010000:00010000:1.0:1789762963.899238:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762963.899242:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05d31 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762963.899248:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05d31 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3d:0x0].0x77289514 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762963.900438:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762963.900466:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015160960 00000100:00100000:3.0:1789762963.900471:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015160960 00000100:00100000:3.0:1789762963.900524:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 127 00000100:00100000:3.0:1789762963.900529:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:15854:x1876702015160960:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762963.900603:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15854:x1876702015160960:12345-192.168.203.48@tcp:35:rm.0 Request processed in 73us (166us total) trans 4294967636 rc 0/0 00000100:00100000:3.0:1789762963.900608:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 127 00000100:00100000:0.0:1789762963.901276:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762963.901312:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015161216 00000100:00100000:2.0:1789762963.901319:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015161216 00000100:00100000:2.0:1789762963.901351:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 388 00000100:00100000:2.0:1789762963.901359:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15854:x1876702015161216:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:2.0:1789762963.901381:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789762963.901393:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05d00 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3d:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd92d expref: 29 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901410:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05bf6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd8a1 expref: 28 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901419:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e9272c900) 00010000:00010000:2.0:1789762963.901430:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05b94 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd88c expref: 27 pid: 5806 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901437:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3b00) 00010000:00010000:2.0:1789762963.901450:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762963.901461:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05d46 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901469:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05d46 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901476:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05d46 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901488:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93400/0x416193e012b05d4d lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901495:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93400/0x416193e012b05d4d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901501:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93400/0x416193e012b05d4d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901548:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92800/0x416193e012b05d54 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901557:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92800/0x416193e012b05d54 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901564:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92800/0x416193e012b05d54 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762963.901855:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3d:0x0]: returning size 16384 00010000:00010000:2.0:1789762963.901865:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92800/0x416193e012b05d54 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901873:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92800/0x416193e012b05d54 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901887:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92800/0x416193e012b05d54 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901894:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762963.901898:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92800/0x416193e012b05d54 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901906:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92800/0x416193e012b05d54 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901916:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3d:0x0].0x0 (ffff981e82a26500) 00010000:00010000:2.0:1789762963.901919:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789762963.901922:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92800/0x416193e012b05d54 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901929:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92800/0x416193e012b05d54 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x3d:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901945:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05d46 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901952:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05d46 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901959:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05d46 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901965:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762963.901969:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05d46 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901987:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05d46 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.901996:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26900) 00010000:00010000:2.0:1789762963.901999:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762963.902002:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05d46 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762963.902008:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92000/0x416193e012b05d46 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762963.902020:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e94a61f6 reply state 00000000bae98f88 transno 4294967637 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93400/0x416193e012b05d4d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762963.902028:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789762963.902074:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:15854:x1876702015161216:12345-192.168.203.48@tcp:36:rm.0 Request processed in 714us (797us total) trans 4294967637 rc 0/0 00000100:00100000:2.0:1789762963.902082:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 388 00010000:00010000:3.0:1789762963.902565:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93400/0x416193e012b05d4d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.902572:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93400/0x416193e012b05d4d lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.902580:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93400/0x416193e012b05d4d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.902583:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762963.902585:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93400/0x416193e012b05d4d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.902590:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93400/0x416193e012b05d4d lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.902596:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e82a26a00) 00010000:00010000:3.0:1789762963.902598:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762963.902600:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93400/0x416193e012b05d4d lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762963.902603:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93400/0x416193e012b05d4d lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762964.182653:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762964.182695:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015161344 00000100:00100000:3.0:1789762964.182705:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015161344 00000100:00100000:3.0:1789762964.182754:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 389 00000100:00100000:3.0:1789762964.182766:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+27:15865:x1876702015161344:12345-192.168.203.48@tcp:101:mkdir.0 00010000:00010000:3.0:1789762964.182787:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762964.182805:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d5b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd934 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.182859:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d62 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.182874:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d62 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.182883:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d62 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762964.182911:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762964.182914:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762964.182929:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d62 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.182949:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05d5b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd934 expref: 29 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.182963:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d62 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd934 expref: 28 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.182970:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:3.0:1789762964.182973:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762964.182975:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05200, rc 0) 00000100:00100000:3.0:1789762964.183025:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:15865:x1876702015161344:12345-192.168.203.48@tcp:101:mkdir.0 Request processed in 263us (378us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762964.183031:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 389 00000100:00100000:3.0:1789762964.184137:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762964.184199:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015161600 00000100:00100000:0.0:1789762964.184205:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015161600 00000100:00100000:0.0:1789762964.184250:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 390 00000100:00100000:0.0:1789762964.184263:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:15865:x1876702015161600:12345-192.168.203.48@tcp:36:mkdir.0 00010000:00010000:0.0:1789762964.184296:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789762964.184314:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05d62 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd934 expref: 28 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.184331:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:0.0:1789762964.184341:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762964.184392:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05d69 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.184400:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05d69 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.184406:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05d69 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.184417:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05d70 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.184422:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05d70 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.184427:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05d70 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762964.185028:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x41:0x0]: returning size 16384 00010000:00010000:0.0:1789762964.185035:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05d69 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.185042:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05d69 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.185046:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05d69 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.185050:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762964.185053:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05d69 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.185059:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05d69 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.185066:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3000) 00010000:00010000:0.0:1789762964.185068:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762964.185070:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05d69 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.185075:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3e00/0x416193e012b05d69 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789762964.185105:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000412ca12b reply state 000000001a46abdf transno 4294967638 ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05d70 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762964.185112:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789762964.185169:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15865:x1876702015161600:12345-192.168.203.48@tcp:36:mkdir.0 Request processed in 910us (1033us total) trans 4294967638 rc 0/0 00000100:00100000:0.0:1789762964.185179:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 390 00010000:00010000:2.0:1789762964.185614:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05d70 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.185626:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05d70 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.185634:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05d70 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.185641:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2000 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762964.185645:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05d70 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.185666:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05d70 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.185675:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3800) 00010000:00010000:2.0:1789762964.185677:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762964.185680:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05d70 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.185684:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2000/0x416193e012b05d70 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.189997:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762964.190060:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015161728 00000100:00100000:2.0:1789762964.190066:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015161728 00000100:00100000:2.0:1789762964.190118:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 391 00000100:00100000:2.0:1789762964.190128:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:15866:x1876702015161728:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:2.0:1789762964.190139:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762964.190166:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92600/0x416193e012b05d77 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd93b expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.190196:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93000/0x416193e012b05d7e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.190203:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93000/0x416193e012b05d7e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.190209:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93000/0x416193e012b05d7e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762964.190223:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789762964.190231:0:8165:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789762964.190244:0:8165:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93000/0x416193e012b05d7e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.190260:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92600/0x416193e012b05d77 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd93b expref: 30 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.190272:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93000/0x416193e012b05d7e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd93b expref: 29 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.190277:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26500) 00010000:00010000:2.0:1789762964.190279:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789762964.190282:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfc93000, rc 0) 00000100:00100000:2.0:1789762964.190314:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15866:x1876702015161728:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 189us (320us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762964.190320:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 391 00000100:00100000:3.0:1789762964.191291:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762964.191324:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015161984 00000100:00100000:0.0:1789762964.191329:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015161984 00000100:00100000:0.0:1789762964.191358:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 392 00000100:00100000:0.0:1789762964.191366:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15866:x1876702015161984:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:0.0:1789762964.191375:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789762964.191377:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1789762964.191387:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93000/0x416193e012b05d7e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd93b expref: 29 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191399:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e82a26500) 00010000:00010000:0.0:1789762964.191404:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789762964.191413:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3000/0x416193e012b05d85 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd942 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191474:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05d8c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191481:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05d8c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191487:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05d8c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191499:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05d93 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191504:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05d93 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191513:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05d93 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191565:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05d9a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191574:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05d9a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191583:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05d9a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789762964.191592:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x41:0x0]: returning size 16384 00010000:00010000:0.0:1789762964.191603:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05d9a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191610:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05d9a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191616:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05d9a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191622:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762964.191626:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05d9a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191633:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05d9a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191643:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981ea0aa3500) 00010000:00010000:0.0:1789762964.191646:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762964.191650:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05d9a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191657:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2400/0x416193e012b05d9a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191670:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05d8c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191677:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05d8c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191683:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05d8c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191688:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762964.191691:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05d8c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191697:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05d8c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191704:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3000) 00010000:00010000:0.0:1789762964.191707:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762964.191710:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05d8c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191716:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2600/0x416193e012b05d8c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191724:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05d93 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191730:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05d93 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191737:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05d93 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191742:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:0.0:1789762964.191746:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05d93 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191752:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05d93 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191759:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981ea0aa3300) 00010000:00010000:0.0:1789762964.191761:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762964.191764:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05d93 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191770:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d2a00/0x416193e012b05d93 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191791:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3000/0x416193e012b05d85 lrc: 1/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd942 expref: 29 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191800:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981ea0aa3000) 00010000:00010000:0.0:1789762964.191803:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789762964.191807:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ec21d3000/0x416193e012b05d85 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd942 expref: 29 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.191820:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ec21d3000, rc 0) 00000100:00100000:0.0:1789762964.191857:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:15866:x1876702015161984:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 491us (566us total) trans 0 rc 301/301 00000100:00100000:0.0:1789762964.191869:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 392 00000100:00100000:0.0:1789762964.192627:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762964.192659:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015162112 00000100:00100000:3.0:1789762964.192664:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015162112 00000100:00100000:3.0:1789762964.192692:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 393 00000100:00100000:3.0:1789762964.192701:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+28:15866:x1876702015162112:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:3.0:1789762964.192709:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762964.192723:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05da1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd949 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.192751:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b05da8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.192758:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b05da8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.192764:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b05da8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762964.192778:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789762964.192780:0:5805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762964.192789:0:5805:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b05da8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.192803:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05da1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd949 expref: 30 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.192814:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b05da8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd949 expref: 29 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.192819:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762964.192821:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762964.192823:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec04800, rc 0) 00000100:00100000:3.0:1789762964.192854:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15866:x1876702015162112:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 155us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762964.192860:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 393 00000100:00100000:3.0:1789762964.193611:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762964.193644:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015162368 00000100:00100000:2.0:1789762964.193652:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015162368 00000100:00100000:2.0:1789762964.193690:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 394 00000100:00100000:2.0:1789762964.193701:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+29:15866:x1876702015162368:12345-192.168.203.48@tcp:101:lfs.0 00010000:00010000:2.0:1789762964.193711:0:8165:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762964.193724:0:8165:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05daf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd950 expref: -99 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193762:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93e00/0x416193e012b05db6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193770:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93e00/0x416193e012b05db6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193786:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93e00/0x416193e012b05db6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193796:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92e00/0x416193e012b05dbd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193801:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92e00/0x416193e012b05dbd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193806:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92e00/0x416193e012b05dbd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762964.193857:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x41:0x0]: returning size 16384 00010000:00010000:2.0:1789762964.193904:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93e00/0x416193e012b05db6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193909:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93e00/0x416193e012b05db6 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193914:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93e00/0x416193e012b05db6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193917:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762964.193920:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93e00/0x416193e012b05db6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193924:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93e00/0x416193e012b05db6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193931:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:2.0:1789762964.193933:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762964.193941:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93e00/0x416193e012b05db6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193945:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc93e00/0x416193e012b05db6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193953:0:8165:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92e00/0x416193e012b05dbd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193957:0:8165:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92e00/0x416193e012b05dbd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193962:0:8165:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92e00/0x416193e012b05dbd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193965:0:8165:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762964.193967:0:8165:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92e00/0x416193e012b05dbd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193971:0:8165:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92e00/0x416193e012b05dbd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193977:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e82a26c00) 00010000:00010000:2.0:1789762964.193979:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789762964.193980:0:8165:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92e00/0x416193e012b05dbd lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.193984:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92e00/0x416193e012b05dbd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.194005:0:8165:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05daf lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd950 expref: 31 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.194020:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:2.0:1789762964.194022:0:8165:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789762964.194025:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05daf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd950 expref: 31 pid: 8165 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.194032:0:8165:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981ebfc92a00, rc 0) 00000100:00100000:2.0:1789762964.194063:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:15866:x1876702015162368:12345-192.168.203.48@tcp:101:lfs.0 Request processed in 367us (453us total) trans 4294967639 rc 301/301 00000100:00100000:2.0:1789762964.194069:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 394 00000100:00100000:0.0:1789762964.194940:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762964.194964:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015162624 00000100:00100000:3.0:1789762964.194968:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015162624 00000100:00100000:3.0:1789762964.194993:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 395 00000100:00100000:3.0:1789762964.195000:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+30:15866:x1876702015162624:12345-192.168.203.48@tcp:36:lfs.0 00010000:00010000:3.0:1789762964.195033:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05dcb lrc: 3/0,1 mode: --/PW res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.195041:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05dcb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.195048:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05dcb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762964.195194:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cf1f6ea9 reply state 000000006f260260 transno 4294967640 ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05dcb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.195203:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:3.0:1789762964.195209:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x41:0x0]: returning size 16384 00000100:00100000:3.0:1789762964.195255:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:15866:x1876702015162624:12345-192.168.203.48@tcp:36:lfs.0 Request processed in 255us (314us total) trans 4294967640 rc 0/0 00000100:00100000:3.0:1789762964.195262:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 395 00010000:00010000:3.0:1789762964.195789:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05dcb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.195796:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05dcb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.195801:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05dcb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.195806:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec05c00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762964.195809:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05dcb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.195858:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05dcb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.195870:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:3.0:1789762964.195874:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789762964.195878:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05dcb lrc: 2/0,0 mode: --/PW res: [0x200000401:0x41:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.195885:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05dcb lrc: 0/0,0 mode: --/PW res: [0x200000401:0x41:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762964.195919:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762964.195960:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015162880 00000100:00100000:2.0:1789762964.195967:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015162880 00000100:00100000:2.0:1789762964.196006:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 128 00000100:00100000:2.0:1789762964.196016:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+31:15866:x1876702015162880:12345-192.168.203.48@tcp:35:lfs.0 00000100:00100000:2.0:1789762964.196175:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:15866:x1876702015162880:12345-192.168.203.48@tcp:35:lfs.0 Request processed in 160us (256us total) trans 4294967641 rc 0/0 00000100:00100000:2.0:1789762964.196182:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 128 00000100:00100000:0.0:1789762964.199798:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762964.199828:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015163008 00000100:00100000:3.0:1789762964.199833:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015163008 00000100:00100000:3.0:1789762964.199861:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 396 00000100:00100000:3.0:1789762964.199872:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+32:15867:x1876702015163008:12345-192.168.203.48@tcp:101:chmod.0 00010000:00010000:3.0:1789762964.199884:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762964.199894:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05dd2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd957 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.199928:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05dd9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.199936:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05dd9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.199942:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05dd9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.199951:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05de0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.199957:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05de0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.199961:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05de0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.199993:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05de7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.199998:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05de7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200003:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05de7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762964.200015:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x41:0x0]: returning size 16384 00000004:00200000:3.0:1789762964.200017:0:5805:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762964.200025:0:5805:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05de7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200031:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05dd9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200036:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05dd9 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200040:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05dd9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200047:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec05200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762964.200049:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05dd9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200053:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05dd9 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200058:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762964.200060:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762964.200062:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05dd9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200066:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05dd9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200072:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05de0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200076:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05de0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200081:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05de0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200104:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec05400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762964.200107:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05de0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200111:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05de0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200120:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272c200) 00010000:00010000:3.0:1789762964.200122:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762964.200124:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05de0 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200129:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05de0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200160:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05dd2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd957 expref: 34 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200169:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05de7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd957 expref: 33 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.200174:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e9272c800) 00010000:00010000:3.0:1789762964.200176:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762964.200178:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec04a00, rc 0) 00000100:00100000:3.0:1789762964.200212:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:15867:x1876702015163008:12345-192.168.203.48@tcp:101:chmod.0 Request processed in 343us (416us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762964.200220:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 396 00000100:00100000:0.0:1789762964.201117:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762964.201157:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015163264 00000100:00100000:2.0:1789762964.201163:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015163264 00000100:00100000:2.0:1789762964.201191:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 397 00000100:00100000:2.0:1789762964.201200:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:15867:x1876702015163264:12345-192.168.203.48@tcp:36:chmod.0 00010000:00010000:2.0:1789762964.201220:0:8165:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789762964.201231:0:8165:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05de7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x41:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd957 expref: 33 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.201242:0:8165:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e9272c800) 00010000:00010000:2.0:1789762964.201255:0:8165:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789762964.201266:0:8165:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05dee lrc: 3/0,1 mode: --/PW res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.201273:0:8165:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05dee lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.201279:0:8165:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05dee lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789762964.201374:0:8165:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005f923a4e reply state 000000003475daf7 transno 4294967642 ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05dee lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762964.201381:0:8165:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789762964.201385:0:8165:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x41:0x0]: returning size 16384 00000100:00100000:2.0:1789762964.201427:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:15867:x1876702015163264:12345-192.168.203.48@tcp:36:chmod.0 Request processed in 228us (310us total) trans 4294967642 rc 0/0 00000100:00100000:2.0:1789762964.201434:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 397 00010000:00010000:2.0:1789762964.201934:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05dee lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.201943:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05dee lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.201948:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05dee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.201953:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762964.201955:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05dee lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.201960:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05dee lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.201974:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e82a26c00) 00010000:00010000:2.0:1789762964.201977:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789762964.201980:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05dee lrc: 2/0,0 mode: --/PW res: [0x200000401:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.201984:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981ebfc92a00/0x416193e012b05dee lrc: 0/0,0 mode: --/PW res: [0x200000401:0x41:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 8165 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762964.220112:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762964.220207:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015163392 00000100:00100000:3.0:1789762964.220215:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015163392 00000100:00100000:3.0:1789762964.220250:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 398 00000100:00100000:3.0:1789762964.220257:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+33:15873:x1876702015163392:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762964.220271:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762964.220282:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05df5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd95e expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220322:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05dfc lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220331:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05dfc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220338:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05dfc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220347:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05e03 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220352:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05e03 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220356:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05e03 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220412:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05e0a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220418:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05e0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220422:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05e0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762964.220434:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x41:0x0]: returning size 16384 00000004:00200000:3.0:1789762964.220437:0:6611:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762964.220447:0:6611:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05e0a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220457:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05dfc lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220461:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05dfc lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220466:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05dfc lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220469:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec05400 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762964.220472:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05dfc lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220477:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05dfc lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220483:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762964.220485:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762964.220488:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05dfc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220492:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05400/0x416193e012b05dfc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220498:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05e03 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220502:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05e03 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220506:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05e03 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220510:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8ec05200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762964.220512:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05e03 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220516:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05e03 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220520:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272c200) 00010000:00010000:3.0:1789762964.220521:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762964.220523:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05e03 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220526:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05200/0x416193e012b05e03 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220543:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04e00/0x416193e012b05df5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd95e expref: 35 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220551:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05e0a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd95e expref: 34 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.220556:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:3.0:1789762964.220557:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789762964.220558:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e8ec05c00, rc 0) 00000100:00100000:3.0:1789762964.220596:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:15873:x1876702015163392:12345-192.168.203.48@tcp:101:dd.500 Request processed in 339us (510us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762964.220604:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 398 00000100:00100000:0.0:1789762964.221840:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762964.221873:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015163648 00000100:00100000:3.0:1789762964.221878:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015163648 00000100:00100000:3.0:1789762964.221901:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 399 00000100:00100000:3.0:1789762964.221906:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+34:15873:x1876702015163648:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762964.221914:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762964.221921:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05e11 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd965 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.221983:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05e18 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.221990:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05e18 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.221995:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05e18 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222004:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05e1f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222009:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05e1f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222013:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05e1f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789762964.222333:0:5805:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1b:0x0], index 0: 0 00010000:00010000:3.0:1789762964.222468:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05e26 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x43:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222474:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05e26 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222483:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05e26 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222503:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05e18 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222508:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05e18 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222512:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05e18 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222515:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762964.222517:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05e18 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222521:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05e18 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222526:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:3.0:1789762964.222527:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762964.222529:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05e18 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222533:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab200/0x416193e012b05e18 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762964.222539:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cfd2adcf reply state 0000000099c3cbc2 transno 4294967643 ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05e1f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.222545:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789762964.222559:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04a00/0x416193e012b05e11 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd965 expref: 37 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222567:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05e26 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x43:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd965 expref: 36 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.222572:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x43:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:3.0:1789762964.222573:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789762964.222575:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aba00, rc 0) 00000100:00100000:3.0:1789762964.222606:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+37:15873:x1876702015163648:12345-192.168.203.48@tcp:101:dd.500 Request processed in 698us (767us total) trans 4294967643 rc 0/0 00000100:00100000:3.0:1789762964.222613:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 399 00010000:00010000:1.0:1789762964.223250:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05e1f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.223262:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05e1f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.223268:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05e1f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.223272:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aac00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762964.223275:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05e1f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.223280:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05e1f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.223290:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x4009f790 (ffff981e9272c800) 00010000:00010000:1.0:1789762964.223293:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762964.223296:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05e1f lrc: 2/0,0 mode: --/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.223300:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aac00/0x416193e012b05e1f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762964.225956:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762964.226022:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015163776 00000100:00100000:1.0:1789762964.226028:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015163776 00000100:00100000:1.0:1789762964.226066:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487054401664 time 487054456497 state 0x0 ref 1 00000100:00100000:1.0:1789762964.226077:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487104456497 time 487054456497 state 0x1 ref 1 00000100:00100000:1.0:1789762964.226132:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 2 deadline 487104456497 time 487530039422 state 0x0 ref 1 00000100:00100000:1.0:1789762964.226158:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac6c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+38:2363:x1876702015163776:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762964.226192:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789762964.226200:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05e34 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.226213:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=487s v=(0 0) 00010000:00010000:1.0:1789762964.226218:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762964.226224:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05e34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.226236:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05e34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762964.245864:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968264 is committed 00002000:00010000:3.0:1789762964.245875:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:3.0:1789762964.245880:0:12582:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb mtime from disk: 0 -> 1789762964 00002000:00010000:3.0:1789762964.245883:0:12582:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb ctime from disk: 0 -> 1789762964 00002000:00010000:3.0:1789762964.245885:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 1 -> 2051 00010000:00010000:3.0:1789762964.245898:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05e34 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762964.245902:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968268 is committed 00010000:00010000:3.0:1789762964.245911:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05e34 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.245919:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05e34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.245925:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.245929:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05e34 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.245940:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05e34 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.245958:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05e34 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.245963:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05e34 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.246041:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac6c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015163776:12345-192.168.203.48@tcp:4:dd.500 Request processed in 19896us (20082us total) trans 4294968268 rc 0/0 00000100:00100000:3.0:1789762964.246049:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1388 00000100:00100000:3.0:1789762964.246059:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 2 deadline 487104456497 time 487530039422 state 0x2 ref 0 00000100:00100000:0.0:1789762964.258902:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762964.258955:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015163904 00000100:00100000:2.0:1789762964.258963:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015163904 00000100:00100000:2.0:1789762964.259006:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 2 deadline 487104456497 time 487530039422 state 0x0 ref 1 00000100:00100000:2.0:1789762964.259017:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 2 deadline 487580039422 time 487530039422 state 0x1 ref 1 00000100:00100000:2.0:1789762964.259040:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 1 deadline 487580039422 time 487562949569 state 0x0 ref 1 00000100:00100000:2.0:1789762964.259050:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015163904:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762964.259116:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 1048576, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x803 00010000:00010000:2.0:1789762964.259123:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e3b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.259135:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=487s v=(0 0) 00010000:00010000:2.0:1789762964.259138:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762964.259157:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.259177:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762964.270616:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968269 is committed 00002000:00010000:2.0:1789762964.270619:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 1048576 -> 2097152 00002000:00010000:2.0:1789762964.270624:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 2051 -> 4103 00010000:00010000:2.0:1789762964.270633:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e3b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.270643:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e3b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.270649:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.270655:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762964.270658:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e3b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.270664:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e3b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.270677:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e3b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.270681:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e3b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762964.270749:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015163904:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11696us (11847us total) trans 4294968269 rc 0/0 00000100:00100000:2.0:1789762964.270761:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1389 00000100:00100000:2.0:1789762964.270771:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 1 deadline 487580039422 time 487562949569 state 0x2 ref 0 00000100:00100000:3.0:1789762964.272795:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762964.272834:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015164032 00000100:00100000:2.0:1789762964.272841:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015164032 00000100:00100000:2.0:1789762964.272868:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 1 deadline 487580039422 time 487562949569 state 0x0 ref 1 00000100:00100000:2.0:1789762964.272878:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 1 deadline 487612949569 time 487562949569 state 0x1 ref 1 00000100:00100000:2.0:1789762964.272900:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487612949569 time 487576809268 state 0x0 ref 1 00000100:00100000:2.0:1789762964.272908:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015164032:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762964.272929:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 2097152, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x1007 00010000:00010000:2.0:1789762964.272934:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b05e42 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.272943:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=487s v=(0 0) 00010000:00010000:2.0:1789762964.272946:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762964.272950:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b05e42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.272958:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b05e42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762964.284661:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968270 is committed 00002000:00010000:3.0:1789762964.284662:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 2097152 -> 3145728 00002000:00010000:3.0:1789762964.284668:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 4103 -> 6151 00010000:00010000:3.0:1789762964.284677:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b05e42 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.284686:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b05e42 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.284693:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b05e42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.284702:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.284705:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b05e42 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.284711:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b05e42 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.284720:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b05e42 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.284724:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b05e42 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.284768:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfa300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015164032:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11859us (11973us total) trans 4294968270 rc 0/0 00000100:00100000:3.0:1789762964.284775:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1390 00000100:00100000:3.0:1789762964.284783:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487612949569 time 487576809268 state 0x2 ref 0 00000100:00100000:3.0:1789762964.286613:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762964.286670:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015164160 00000100:00100000:1.0:1789762964.286677:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015164160 00000100:00100000:1.0:1789762964.286708:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487612949569 time 487576809268 state 0x0 ref 1 00000100:00100000:1.0:1789762964.286718:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487626809268 time 487576809268 state 0x1 ref 1 00000100:00100000:3.0:1789762964.323002:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487626809268 time 487626904425 state 0x0 ref 1 00000100:00100000:3.0:1789762964.323018:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015164160:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762964.323051:0:17497:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 3145728, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x1807 00010000:00010000:3.0:1789762964.323057:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e49 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.323069:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=487s v=(0 0) 00010000:00010000:3.0:1789762964.323072:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762964.323076:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.323103:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762964.335648:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968271 is committed 00002000:00010000:3.0:1789762964.335701:0:17497:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 3145728 -> 4194304 00002000:00010000:3.0:1789762964.335706:0:17497:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 6151 -> 8199 00010000:00010000:3.0:1789762964.335714:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e49 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.335723:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.335729:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.335734:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aaa00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.335738:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.335744:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.335761:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.335766:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.335819:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015164160:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12804us (49206us total) trans 4294968271 rc 0/0 00000100:00100000:3.0:1789762964.335827:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1391 00000100:00100000:3.0:1789762964.335835:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487626809268 time 487626904425 state 0x2 ref 0 00000100:00100000:0.0:1789762964.338050:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762964.338100:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015164288 00000100:00100000:3.0:1789762964.338108:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015164288 00000100:00100000:3.0:1789762964.338158:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487626809268 time 487626904425 state 0x0 ref 1 00000100:00100000:3.0:1789762964.338169:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487676904425 time 487626904425 state 0x1 ref 1 00000100:00100000:0.0:1789762964.373155:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487676904425 time 487677039071 state 0x0 ref 1 00000100:00100000:0.0:1789762964.373179:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015164288:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762964.373227:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 4194304, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x2007 00010000:00010000:0.0:1789762964.373234:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05e50 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.373247:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=487s v=(0 0) 00010000:00010000:0.0:1789762964.373251:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762964.373255:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05e50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762964.373266:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05e50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762964.385444:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 4194304 -> 5242880 00000001:00080000:2.0:1789762964.385444:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968272 is committed 00002000:00010000:1.0:1789762964.385451:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 8199 -> 10247 00010000:00010000:1.0:1789762964.385461:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05e50 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.385470:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05e50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.385477:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05e50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.385482:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762964.385486:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05e50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.385492:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05e50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.385505:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05e50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.385515:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05e50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762964.385571:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015164288:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12400us (47521us total) trans 4294968272 rc 0/0 00000100:00100000:1.0:1789762964.385579:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1392 00000100:00100000:1.0:1789762964.385587:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487676904425 time 487677039071 state 0x2 ref 0 00000100:00100000:3.0:1789762964.387679:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762964.387739:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015164416 00000100:00100000:2.0:1789762964.387749:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015164416 00000100:00100000:2.0:1789762964.387787:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487676904425 time 487677039071 state 0x0 ref 1 00000100:00100000:2.0:1789762964.387798:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487727039071 time 487677039071 state 0x1 ref 1 00010000:00010000:0.0:1789762964.407197:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762964.423199:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487727039071 time 487727098274 state 0x0 ref 1 00000100:00100000:2.0:1789762964.423216:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015164416:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762964.423251:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 5242880, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x2807 00010000:00010000:2.0:1789762964.423257:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e57 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.423269:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=487s v=(0 0) 00010000:00010000:2.0:1789762964.423272:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762964.423277:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.423287:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762964.435469:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968273 is committed 00002000:00010000:3.0:1789762964.435529:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 5242880 -> 6291456 00002000:00010000:3.0:1789762964.435534:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 10247 -> 12295 00010000:00010000:3.0:1789762964.435544:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e57 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.435553:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e57 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.435563:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.435569:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.435572:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e57 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.435578:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e57 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.435590:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e57 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.435595:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05e57 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.435654:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015164416:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12441us (47974us total) trans 4294968273 rc 0/0 00000100:00100000:3.0:1789762964.435665:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1393 00000100:00100000:3.0:1789762964.435677:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487727039071 time 487727098274 state 0x2 ref 0 00000100:00100000:3.0:1789762964.437693:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762964.437737:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015164544 00000100:00100000:1.0:1789762964.437745:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015164544 00000100:00100000:1.0:1789762964.437780:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487727039071 time 487727098274 state 0x0 ref 1 00000100:00100000:1.0:1789762964.437791:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487777098274 time 487727098274 state 0x1 ref 1 00000100:00100000:1.0:1789762964.473280:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487777098274 time 487777171213 state 0x0 ref 1 00000100:00100000:1.0:1789762964.473307:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015164544:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762964.473359:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 6291456, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x3007 00010000:00010000:1.0:1789762964.473369:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e5e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.473387:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=487s v=(0 0) 00010000:00010000:1.0:1789762964.473391:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762964.473400:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.473412:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762964.485378:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968274 is committed 00002000:00010000:1.0:1789762964.485386:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 6291456 -> 7340032 00002000:00010000:1.0:1789762964.485393:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 12295 -> 14343 00010000:00010000:1.0:1789762964.485405:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e5e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.485418:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e5e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.485428:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.485436:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762964.485442:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e5e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.485453:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e5e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.485471:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e5e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.485479:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e5e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762964.485544:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015164544:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12243us (47850us total) trans 4294968274 rc 0/0 00000100:00100000:1.0:1789762964.485555:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1394 00000100:00100000:1.0:1789762964.485566:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487777098274 time 487777171213 state 0x2 ref 0 00000100:00100000:2.0:1789762964.487625:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762964.487671:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015164672 00000100:00100000:3.0:1789762964.487680:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015164672 00000100:00100000:3.0:1789762964.487716:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487777098274 time 487777171213 state 0x0 ref 1 00000100:00100000:3.0:1789762964.487728:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487827171213 time 487777171213 state 0x1 ref 1 00000100:00100000:3.0:1789762964.523331:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487827171213 time 487827229324 state 0x0 ref 1 00000100:00100000:3.0:1789762964.523353:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81397800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015164672:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762964.523396:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 7340032, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x3807 00010000:00010000:3.0:1789762964.523404:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e65 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.523417:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=487s v=(0 0) 00010000:00010000:3.0:1789762964.523420:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762964.523425:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.523436:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762964.533968:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968275 is committed 00002000:00010000:3.0:1789762964.533975:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 7340032 -> 8388608 00002000:00010000:3.0:1789762964.533981:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 14343 -> 16391 00010000:00010000:3.0:1789762964.533991:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e65 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.534000:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e65 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.534011:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.534016:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aaa00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.534019:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e65 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.534025:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e65 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.534039:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e65 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.534044:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e65 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.534121:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81397800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015164672:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10775us (46496us total) trans 4294968275 rc 0/0 00000100:00100000:3.0:1789762964.534130:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1395 00000100:00100000:3.0:1789762964.534138:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487827171213 time 487827229324 state 0x2 ref 0 00000100:00100000:0.0:1789762964.536473:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762964.536554:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015164800 00000100:00100000:2.0:1789762964.536563:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015164800 00000100:00100000:2.0:1789762964.536598:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487827171213 time 487827229324 state 0x0 ref 1 00000100:00100000:2.0:1789762964.536609:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487877229324 time 487827229324 state 0x1 ref 1 00000100:00100000:2.0:1789762964.573414:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487877229324 time 487877304203 state 0x0 ref 1 00000100:00100000:2.0:1789762964.573439:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015164800:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762964.573485:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 8388608, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x4007 00010000:00010000:2.0:1789762964.573496:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b05e6c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.573509:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=487s v=(0 0) 00010000:00010000:2.0:1789762964.573513:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762964.573518:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b05e6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.573529:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b05e6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762964.584488:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968276 is committed 00002000:00010000:3.0:1789762964.584517:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 8388608 -> 9437184 00002000:00010000:3.0:1789762964.584523:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 16391 -> 18439 00010000:00010000:3.0:1789762964.584536:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b05e6c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.584555:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b05e6c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.584565:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b05e6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.584573:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.584578:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b05e6c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.584589:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b05e6c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.584618:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b05e6c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.584628:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b05e6c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.584703:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015164800:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11271us (48228us total) trans 4294968276 rc 0/0 00000100:00100000:3.0:1789762964.584727:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1396 00000100:00100000:3.0:1789762964.584741:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487877229324 time 487877304203 state 0x2 ref 0 00000100:00100000:0.0:1789762964.587310:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762964.587355:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015164928 00000100:00100000:1.0:1789762964.587363:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015164928 00000100:00100000:1.0:1789762964.587401:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487877229324 time 487877304203 state 0x0 ref 1 00000100:00100000:1.0:1789762964.587412:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487927304203 time 487877304203 state 0x1 ref 1 00000100:00100000:1.0:1789762964.623468:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487927304203 time 487927365587 state 0x0 ref 1 00000100:00100000:1.0:1789762964.623492:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015164928:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762964.623542:0:14913:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 9437184, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x4807 00010000:00010000:1.0:1789762964.623551:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e73 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.623568:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=487s v=(0 0) 00010000:00010000:1.0:1789762964.623574:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762964.623579:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.623597:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762964.635154:0:14913:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 9437184 -> 10485760 00002000:00010000:3.0:1789762964.635162:0:14913:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 18439 -> 20487 00000001:00080000:2.0:1789762964.635165:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968277 is committed 00010000:00010000:3.0:1789762964.635179:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e73 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.635195:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e73 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.635204:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.635209:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.635213:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e73 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.635220:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e73 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.635235:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e73 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.635240:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05e73 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.635311:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015164928:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11825us (47999us total) trans 4294968277 rc 0/0 00000100:00100000:3.0:1789762964.635320:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1397 00000100:00100000:3.0:1789762964.635330:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487927304203 time 487927365587 state 0x2 ref 0 00000100:00100000:3.0:1789762964.637957:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762964.638001:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015165056 00000100:00100000:2.0:1789762964.638016:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015165056 00000100:00100000:2.0:1789762964.638055:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487927304203 time 487927365587 state 0x0 ref 1 00000100:00100000:2.0:1789762964.638067:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487977365587 time 487927365587 state 0x1 ref 1 00000100:00100000:3.0:1789762964.673564:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487977365587 time 487977460778 state 0x0 ref 1 00000100:00100000:3.0:1789762964.673595:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015165056:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762964.673649:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 10485760, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x5007 00010000:00010000:3.0:1789762964.673657:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e7a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.673670:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=487s v=(0 0) 00010000:00010000:3.0:1789762964.673674:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762964.673678:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.673689:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762964.684615:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968278 is committed 00002000:00010000:1.0:1789762964.684617:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 10485760 -> 11534336 00002000:00010000:1.0:1789762964.684623:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 20487 -> 22535 00010000:00010000:1.0:1789762964.684633:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e7a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.684643:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e7a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.684650:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.684655:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aaa00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762964.684659:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e7a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.684665:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e7a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.684680:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e7a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.684685:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000087ea9cad/0x416193e012b05e7a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762964.684747:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015165056:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11165us (46789us total) trans 4294968278 rc 0/0 00000100:00100000:1.0:1789762964.684756:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1398 00000100:00100000:1.0:1789762964.684765:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487977365587 time 487977460778 state 0x2 ref 0 00000100:00100000:0.0:1789762964.686966:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762964.687014:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015165184 00000100:00100000:2.0:1789762964.687022:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015165184 00000100:00100000:2.0:1789762964.687060:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 487977365587 time 487977460778 state 0x0 ref 1 00000100:00100000:2.0:1789762964.687071:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488027460778 time 487977460778 state 0x1 ref 1 00000100:00100000:2.0:1789762964.723616:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488027460778 time 488027514167 state 0x0 ref 1 00000100:00100000:2.0:1789762964.723635:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015165184:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762964.723678:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 11534336, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x5807 00010000:00010000:2.0:1789762964.723685:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05e81 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.723698:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:2.0:1789762964.723701:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762964.723706:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05e81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.723718:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05e81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762964.734012:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968279 is committed 00002000:00010000:3.0:1789762964.734022:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 11534336 -> 12582912 00002000:00010000:3.0:1789762964.734027:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 22535 -> 24583 00010000:00010000:3.0:1789762964.734038:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05e81 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.734048:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05e81 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.734054:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05e81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.734061:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.734064:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05e81 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.734071:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05e81 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.734084:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05e81 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.734106:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05e81 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.734174:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015165184:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10544us (47209us total) trans 4294968279 rc 0/0 00000100:00100000:3.0:1789762964.734182:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1399 00000100:00100000:3.0:1789762964.734191:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488027460778 time 488027514167 state 0x2 ref 0 00000100:00100000:3.0:1789762964.736179:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762964.736228:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015165312 00000100:00100000:1.0:1789762964.736236:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015165312 00000100:00100000:1.0:1789762964.736273:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488027460778 time 488027514167 state 0x0 ref 1 00000100:00100000:1.0:1789762964.736283:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488077514167 time 488027514167 state 0x1 ref 1 00000100:00100000:1.0:1789762964.773706:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488077514167 time 488077599738 state 0x0 ref 1 00000100:00100000:1.0:1789762964.773736:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015165312:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762964.773790:0:17498:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 12582912, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x6007 00010000:00010000:1.0:1789762964.773797:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e88 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.773811:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:1.0:1789762964.773815:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762964.773820:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.773831:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762964.785387:0:17498:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 12582912 -> 13631488 00000001:00080000:2.0:1789762964.785387:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968280 is committed 00002000:00010000:1.0:1789762964.785396:0:17498:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 24583 -> 26631 00010000:00010000:1.0:1789762964.785409:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e88 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.785421:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e88 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.785431:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.785439:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762964.785444:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e88 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.785454:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e88 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.785488:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e88 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.785498:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e88 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762964.785579:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015165312:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11851us (49395us total) trans 4294968280 rc 0/0 00000100:00100000:1.0:1789762964.785591:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1400 00000100:00100000:1.0:1789762964.785604:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488077514167 time 488077599738 state 0x2 ref 0 00000100:00100000:3.0:1789762964.787855:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762964.787901:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015165440 00000100:00100000:2.0:1789762964.787909:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015165440 00000100:00100000:2.0:1789762964.787956:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488077514167 time 488077599738 state 0x0 ref 1 00000100:00100000:2.0:1789762964.787967:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488127599738 time 488077599738 state 0x1 ref 1 00000100:00100000:2.0:1789762964.823765:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488127599738 time 488127663029 state 0x0 ref 1 00000100:00100000:2.0:1789762964.823788:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015165440:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762964.823833:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 13631488, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x6807 00010000:00010000:2.0:1789762964.823841:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008943d278/0x416193e012b05e8f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.823853:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:2.0:1789762964.823857:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762964.823862:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008943d278/0x416193e012b05e8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.823872:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008943d278/0x416193e012b05e8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762964.834293:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968281 is committed 00002000:00010000:3.0:1789762964.834315:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 13631488 -> 14680064 00002000:00010000:3.0:1789762964.834320:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 26631 -> 28679 00010000:00010000:3.0:1789762964.834331:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008943d278/0x416193e012b05e8f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.834346:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008943d278/0x416193e012b05e8f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.834352:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008943d278/0x416193e012b05e8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.834357:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.834361:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008943d278/0x416193e012b05e8f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.834367:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008943d278/0x416193e012b05e8f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.834382:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008943d278/0x416193e012b05e8f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.834387:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008943d278/0x416193e012b05e8f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.834451:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015165440:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10669us (46594us total) trans 4294968281 rc 0/0 00000100:00100000:3.0:1789762964.834459:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1401 00000100:00100000:3.0:1789762964.834468:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488127599738 time 488127663029 state 0x2 ref 0 00000100:00100000:0.0:1789762964.836649:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762964.836701:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015165568 00000100:00100000:1.0:1789762964.836711:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015165568 00000100:00100000:1.0:1789762964.836754:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488127599738 time 488127663029 state 0x0 ref 1 00000100:00100000:1.0:1789762964.836769:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488177663029 time 488127663029 state 0x1 ref 1 00000100:00100000:2.0:1789762964.873896:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488177663029 time 488177792426 state 0x0 ref 1 00000100:00100000:2.0:1789762964.873935:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015165568:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762964.873995:0:17497:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 14680064, mtime: 0x6aad9d94, atime: 0x0, ctime: 0x6aad9d94, blocks: 0x7007 00010000:00010000:2.0:1789762964.874002:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05e96 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.874015:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:2.0:1789762964.874019:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762964.874024:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05e96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.874035:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05e96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762964.885285:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968282 is committed 00002000:00010000:3.0:1789762964.885306:0:17497:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 14680064 -> 15728640 00002000:00010000:3.0:1789762964.885311:0:17497:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb mtime from disk: 1789762964 -> 1789762965 00002000:00010000:3.0:1789762964.885314:0:17497:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb ctime from disk: 1789762964 -> 1789762965 00002000:00010000:3.0:1789762964.885316:0:17497:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 28679 -> 30727 00010000:00010000:3.0:1789762964.885326:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05e96 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.885336:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05e96 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.885342:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05e96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.885347:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.885351:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05e96 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.885357:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05e96 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.885372:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05e96 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.885377:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05e96 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.885439:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015165568:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11509us (48790us total) trans 4294968282 rc 0/0 00000100:00100000:3.0:1789762964.885450:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1402 00000100:00100000:3.0:1789762964.885459:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488177663029 time 488177792426 state 0x2 ref 0 00000100:00100000:0.0:1789762964.887776:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762964.887816:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015165696 00000100:00100000:3.0:1789762964.887826:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015165696 00000100:00100000:3.0:1789762964.887871:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488177663029 time 488177792426 state 0x0 ref 1 00000100:00100000:3.0:1789762964.887886:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488227792426 time 488177792426 state 0x1 ref 1 00000100:00100000:1.0:1789762964.923988:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488227792426 time 488227888077 state 0x0 ref 1 00000100:00100000:1.0:1789762964.924008:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015165696:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762964.924060:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 15728640, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0x7807 00010000:00010000:1.0:1789762964.924066:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e9d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.924079:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:1.0:1789762964.924083:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762964.924111:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762964.924122:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762964.934563:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968283 is committed 00002000:00010000:3.0:1789762964.934591:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 15728640 -> 16777216 00002000:00010000:3.0:1789762964.934596:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 30727 -> 32775 00010000:00010000:3.0:1789762964.934606:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e9d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.934615:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.934624:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.934633:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.934638:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.934648:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.934669:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.934675:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05e9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.934736:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015165696:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10732us (46959us total) trans 4294968283 rc 0/0 00000100:00100000:3.0:1789762964.934744:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1403 00000100:00100000:3.0:1789762964.934752:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488227792426 time 488227888077 state 0x2 ref 0 00000100:00100000:0.0:1789762964.936869:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762964.936900:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015165824 00000100:00100000:3.0:1789762964.936908:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015165824 00000100:00100000:3.0:1789762964.936943:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488227792426 time 488227888077 state 0x0 ref 1 00000100:00100000:3.0:1789762964.936954:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488277888077 time 488227888077 state 0x1 ref 1 00000100:00100000:2.0:1789762964.974114:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488277888077 time 488277991333 state 0x0 ref 1 00000100:00100000:2.0:1789762964.974155:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015165824:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762964.974203:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 16777216, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0x8007 00010000:00010000:2.0:1789762964.974211:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b05ea4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.974224:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:2.0:1789762964.974227:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762964.974243:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b05ea4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762964.974254:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b05ea4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762964.985270:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968284 is committed 00002000:00010000:3.0:1789762964.985289:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 16777216 -> 17825792 00002000:00010000:3.0:1789762964.985295:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 32775 -> 34823 00010000:00010000:3.0:1789762964.985309:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b05ea4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.985323:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b05ea4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.985335:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b05ea4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.985340:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762964.985344:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b05ea4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.985351:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b05ea4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.985366:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b05ea4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762964.985372:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b05ea4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762964.985434:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015165824:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11303us (48563us total) trans 4294968284 rc 0/0 00000100:00100000:3.0:1789762964.985445:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1404 00000100:00100000:3.0:1789762964.985456:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488277888077 time 488277991333 state 0x2 ref 0 00000100:00100000:0.0:1789762964.987941:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762964.987979:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015165952 00000100:00100000:3.0:1789762964.987987:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015165952 00000100:00100000:3.0:1789762964.988040:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488277888077 time 488277991333 state 0x0 ref 1 00000100:00100000:3.0:1789762964.988055:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488327991333 time 488277991333 state 0x1 ref 1 00000100:00100000:1.0:1789762965.024217:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488327991333 time 488328111221 state 0x0 ref 1 00000100:00100000:1.0:1789762965.024271:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015165952:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:3.0:1789762965.024293:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789762965.024325:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 17825792, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0x8807 00010000:00010000:1.0:1789762965.024335:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05eab lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.024352:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:1.0:1789762965.024356:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762965.024363:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05eab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.024380:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05eab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762965.036549:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968285 is committed 00002000:00010000:1.0:1789762965.036559:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 17825792 -> 18874368 00002000:00010000:1.0:1789762965.036565:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 34823 -> 36871 00010000:00010000:1.0:1789762965.036577:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05eab lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.036588:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05eab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.036595:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05eab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.036603:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762965.036608:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05eab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.036626:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05eab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.036646:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05eab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.036655:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05eab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762965.036725:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015165952:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12489us (48782us total) trans 4294968285 rc 0/0 00000100:00100000:1.0:1789762965.036734:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1405 00000100:00100000:1.0:1789762965.036746:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488327991333 time 488328111221 state 0x2 ref 0 00000100:00100000:0.0:1789762965.039335:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762965.039380:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015166080 00000100:00100000:3.0:1789762965.039388:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015166080 00000100:00100000:3.0:1789762965.039432:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488327991333 time 488328111221 state 0x0 ref 1 00000100:00100000:3.0:1789762965.039444:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488378111221 time 488328111221 state 0x1 ref 1 00000100:00100000:3.0:1789762965.074280:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488378111221 time 488378175990 state 0x0 ref 1 00000100:00100000:3.0:1789762965.074305:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015166080:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762965.074352:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 18874368, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0x9007 00010000:00010000:3.0:1789762965.074359:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.074373:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:3.0:1789762965.074377:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.074382:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.074394:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762965.086132:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968286 is committed 00002000:00010000:3.0:1789762965.086211:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 18874368 -> 19922944 00002000:00010000:3.0:1789762965.086216:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 36871 -> 38919 00010000:00010000:3.0:1789762965.086230:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.086255:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.086266:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.086274:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.086278:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.086288:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.086303:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.086309:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.086379:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015166080:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12080us (47040us total) trans 4294968286 rc 0/0 00000100:00100000:3.0:1789762965.086390:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1406 00000100:00100000:3.0:1789762965.086403:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488378111221 time 488378175990 state 0x2 ref 0 00000100:00100000:2.0:1789762965.089254:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762965.089292:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015166208 00000100:00100000:3.0:1789762965.089300:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015166208 00000100:00100000:3.0:1789762965.089340:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488378111221 time 488378175990 state 0x0 ref 1 00000100:00100000:3.0:1789762965.089351:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488428175990 time 488378175990 state 0x1 ref 1 00000100:00100000:3.0:1789762965.124350:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488428175990 time 488428245142 state 0x0 ref 1 00000100:00100000:3.0:1789762965.124374:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015166208:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762965.124424:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 19922944, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0x9807 00010000:00010000:3.0:1789762965.124432:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.124446:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:3.0:1789762965.124449:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.124454:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.124465:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762965.136013:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968287 is committed 00002000:00010000:3.0:1789762965.136068:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 19922944 -> 20971520 00002000:00010000:3.0:1789762965.136073:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 38919 -> 40967 00010000:00010000:3.0:1789762965.136084:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.136109:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.136116:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.136121:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.136124:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.136131:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.136163:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.136169:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05eb9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.136236:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015166208:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11869us (46990us total) trans 4294968287 rc 0/0 00000100:00100000:3.0:1789762965.136244:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1407 00000100:00100000:3.0:1789762965.136254:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488428175990 time 488428245142 state 0x2 ref 0 00000100:00100000:0.0:1789762965.139424:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762965.139471:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015166336 00000100:00100000:1.0:1789762965.139481:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015166336 00000100:00100000:1.0:1789762965.139530:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488428175990 time 488428245142 state 0x0 ref 1 00000100:00100000:1.0:1789762965.139544:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488478245142 time 488428245142 state 0x1 ref 1 00000100:00100000:3.0:1789762965.174447:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488478245142 time 488478342652 state 0x0 ref 1 00000100:00100000:3.0:1789762965.174474:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015166336:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762965.174521:0:14913:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 20971520, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xa007 00010000:00010000:3.0:1789762965.174530:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.174547:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:3.0:1789762965.174552:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.174560:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.174576:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762965.185781:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968288 is committed 00002000:00010000:3.0:1789762965.185802:0:14913:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 20971520 -> 22020096 00002000:00010000:3.0:1789762965.185808:0:14913:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 40967 -> 43015 00010000:00010000:3.0:1789762965.185820:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.185831:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.185837:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.185842:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.185846:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.185854:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.185868:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.185873:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.185937:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015166336:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11471us (46510us total) trans 4294968288 rc 0/0 00000100:00100000:3.0:1789762965.185949:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1408 00000100:00100000:3.0:1789762965.185960:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488478245142 time 488478342652 state 0x2 ref 0 00000100:00100000:0.0:1789762965.188238:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762965.188285:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015166464 00000100:00100000:2.0:1789762965.188293:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015166464 00000100:00100000:2.0:1789762965.188332:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488478245142 time 488478342652 state 0x0 ref 1 00000100:00100000:2.0:1789762965.188347:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488528342652 time 488478342652 state 0x1 ref 1 00000100:00100000:3.0:1789762965.224518:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488528342652 time 488528416544 state 0x0 ref 1 00000100:00100000:3.0:1789762965.224541:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015166464:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762965.224591:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 22020096, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xa807 00010000:00010000:3.0:1789762965.224598:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.224611:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:3.0:1789762965.224615:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.224620:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.224630:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762965.235115:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968289 is committed 00002000:00010000:3.0:1789762965.235124:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 22020096 -> 23068672 00002000:00010000:3.0:1789762965.235129:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 43015 -> 45063 00010000:00010000:3.0:1789762965.235153:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.235163:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.235170:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.235175:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.235179:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.235185:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.235198:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.235207:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ec7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.235261:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015166464:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10729us (47023us total) trans 4294968289 rc 0/0 00000100:00100000:3.0:1789762965.235268:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1409 00000100:00100000:3.0:1789762965.235276:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488528342652 time 488528416544 state 0x2 ref 0 00000100:00100000:2.0:1789762965.237488:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762965.237521:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015166592 00000100:00100000:3.0:1789762965.237528:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015166592 00000100:00100000:3.0:1789762965.237568:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488528342652 time 488528416544 state 0x0 ref 1 00000100:00100000:3.0:1789762965.237579:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488578416544 time 488528416544 state 0x1 ref 1 00000100:00100000:3.0:1789762965.274604:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488578416544 time 488578500482 state 0x0 ref 1 00000100:00100000:3.0:1789762965.274632:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015166592:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762965.274686:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 23068672, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xb007 00010000:00010000:3.0:1789762965.274696:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ece lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.274714:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:3.0:1789762965.274719:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.274727:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ece lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.274741:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ece lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762965.285871:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968290 is committed 00002000:00010000:3.0:1789762965.285887:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 23068672 -> 24117248 00002000:00010000:3.0:1789762965.285892:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 45063 -> 47111 00010000:00010000:3.0:1789762965.285903:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ece lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.285913:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ece lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.285919:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ece lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.285924:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.285927:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ece lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.285934:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ece lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.285950:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ece lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.285959:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ece lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.286018:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015166592:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11398us (48529us total) trans 4294968290 rc 0/0 00000100:00100000:3.0:1789762965.286026:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1410 00000100:00100000:3.0:1789762965.286036:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488578416544 time 488578500482 state 0x2 ref 0 00000100:00100000:2.0:1789762965.288323:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762965.288368:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015166720 00000100:00100000:1.0:1789762965.288377:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015166720 00000100:00100000:1.0:1789762965.288417:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488578416544 time 488578500482 state 0x0 ref 1 00000100:00100000:1.0:1789762965.288428:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488628500482 time 488578500482 state 0x1 ref 1 00000100:00100000:1.0:1789762965.324651:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488628500482 time 488628549953 state 0x0 ref 1 00000100:00100000:1.0:1789762965.324673:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81397100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015166720:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762965.324718:0:17498:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 24117248, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xb807 00010000:00010000:1.0:1789762965.324725:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05ed5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.324739:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:1.0:1789762965.324742:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762965.324747:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05ed5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.324757:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05ed5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762965.335845:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968291 is committed 00002000:00010000:2.0:1789762965.335851:0:17498:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 24117248 -> 25165824 00002000:00010000:2.0:1789762965.335857:0:17498:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 47111 -> 49159 00010000:00010000:2.0:1789762965.335866:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05ed5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.335876:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05ed5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.335883:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05ed5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.335898:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762965.335902:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05ed5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.335908:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05ed5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.335922:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05ed5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.335927:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05ed5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762965.335984:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81397100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015166720:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11318us (47661us total) trans 4294968291 rc 0/0 00000100:00100000:2.0:1789762965.335992:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1411 00000100:00100000:2.0:1789762965.336001:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488628500482 time 488628549953 state 0x2 ref 0 00000100:00100000:0.0:1789762965.338302:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762965.338343:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015166848 00000100:00100000:3.0:1789762965.338351:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015166848 00000100:00100000:3.0:1789762965.338393:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488628500482 time 488628549953 state 0x0 ref 1 00000100:00100000:3.0:1789762965.338408:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488678549953 time 488628549953 state 0x1 ref 1 00000100:00100000:2.0:1789762965.374778:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488678549953 time 488678675378 state 0x0 ref 1 00000100:00100000:2.0:1789762965.374820:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015166848:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762965.374876:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 25165824, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xc007 00010000:00010000:2.0:1789762965.374884:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05edc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.374909:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:2.0:1789762965.374912:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762965.374917:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05edc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.374928:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05edc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762965.386368:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968292 is committed 00002000:00010000:3.0:1789762965.386390:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 25165824 -> 26214400 00002000:00010000:3.0:1789762965.386396:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 49159 -> 51207 00010000:00010000:3.0:1789762965.386406:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05edc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.386415:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05edc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.386421:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05edc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.386430:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.386434:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05edc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.386440:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05edc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.386453:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05edc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.386458:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c1889e1e/0x416193e012b05edc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.386518:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015166848:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11711us (48214us total) trans 4294968292 rc 0/0 00000100:00100000:3.0:1789762965.386529:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1412 00000100:00100000:3.0:1789762965.386541:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488678549953 time 488678675378 state 0x2 ref 0 00000100:00100000:0.0:1789762965.388889:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762965.388949:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015166976 00000100:00100000:1.0:1789762965.388959:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015166976 00000100:00100000:1.0:1789762965.389005:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488678549953 time 488678675378 state 0x0 ref 1 00000100:00100000:1.0:1789762965.389025:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488728675378 time 488678675378 state 0x1 ref 1 00000100:00100000:3.0:1789762965.424869:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488728675378 time 488728763944 state 0x0 ref 1 00000100:00100000:3.0:1789762965.424893:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015166976:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762965.424943:0:17497:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 26214400, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xc807 00010000:00010000:3.0:1789762965.424951:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ee3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.424965:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:3.0:1789762965.424970:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.424977:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ee3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.424993:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ee3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.431208:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789762965.436764:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968293 is committed 00002000:00010000:3.0:1789762965.436785:0:17497:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 26214400 -> 27262976 00002000:00010000:3.0:1789762965.436791:0:17497:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 51207 -> 53255 00010000:00010000:3.0:1789762965.436802:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ee3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.436812:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ee3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.436818:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ee3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.436823:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.436827:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ee3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.436834:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ee3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.436852:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ee3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.436858:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ee3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.436931:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015166976:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12044us (48039us total) trans 4294968293 rc 0/0 00000100:00100000:3.0:1789762965.436943:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1413 00000100:00100000:3.0:1789762965.436953:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488728675378 time 488728763944 state 0x2 ref 0 00000100:00100000:2.0:1789762965.439374:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762965.439429:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015167104 00000100:00100000:1.0:1789762965.439438:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015167104 00000100:00100000:1.0:1789762965.439486:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488728675378 time 488728763944 state 0x0 ref 1 00000100:00100000:1.0:1789762965.439502:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488778763944 time 488728763944 state 0x1 ref 1 00000100:00100000:1.0:1789762965.474937:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488778763944 time 488778828622 state 0x0 ref 1 00000100:00100000:1.0:1789762965.474974:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81394e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015167104:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762965.475040:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 27262976, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xd007 00010000:00010000:1.0:1789762965.475050:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05eea lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.475067:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:1.0:1789762965.475072:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762965.475079:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05eea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.475122:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05eea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762965.486402:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968294 is committed 00002000:00010000:3.0:1789762965.486429:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 27262976 -> 28311552 00002000:00010000:3.0:1789762965.486437:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 53255 -> 55303 00010000:00010000:3.0:1789762965.486451:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05eea lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.486465:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05eea lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.486483:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05eea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.486502:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.486508:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05eea lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.486525:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05eea lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.486546:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05eea lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.486555:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05eea lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.486643:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81394e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015167104:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11672us (47265us total) trans 4294968294 rc 0/0 00000100:00100000:3.0:1789762965.486654:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1414 00000100:00100000:3.0:1789762965.486666:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488778763944 time 488778828622 state 0x2 ref 0 00000100:00100000:0.0:1789762965.489257:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762965.489305:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015167232 00000100:00100000:3.0:1789762965.489313:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015167232 00000100:00100000:3.0:1789762965.489354:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488778763944 time 488778828622 state 0x0 ref 1 00000100:00100000:3.0:1789762965.489365:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488828828622 time 488778828622 state 0x1 ref 1 00000100:00100000:3.0:1789762965.524984:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488828828622 time 488828879823 state 0x0 ref 1 00000100:00100000:3.0:1789762965.525009:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015167232:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762965.525061:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 28311552, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xd807 00010000:00010000:3.0:1789762965.525069:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ef1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.525082:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:3.0:1789762965.525109:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.525114:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ef1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.525126:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ef1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762965.536320:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968295 is committed 00002000:00010000:3.0:1789762965.536342:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 28311552 -> 29360128 00002000:00010000:3.0:1789762965.536347:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 55303 -> 57351 00010000:00010000:3.0:1789762965.536357:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ef1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.536366:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ef1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.536373:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ef1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.536378:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.536382:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ef1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.536393:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ef1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.536407:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ef1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.536412:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b05ef1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.536470:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015167232:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11471us (47214us total) trans 4294968295 rc 0/0 00000100:00100000:3.0:1789762965.536480:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1415 00000100:00100000:3.0:1789762965.536489:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488828828622 time 488828879823 state 0x2 ref 0 00000100:00100000:2.0:1789762965.538755:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762965.538807:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015167360 00000100:00100000:1.0:1789762965.538816:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015167360 00000100:00100000:1.0:1789762965.538859:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488828828622 time 488828879823 state 0x0 ref 1 00000100:00100000:1.0:1789762965.538875:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488878879823 time 488828879823 state 0x1 ref 1 00000100:00100000:1.0:1789762965.575048:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488878879823 time 488878946040 state 0x0 ref 1 00000100:00100000:1.0:1789762965.575070:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81396680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015167360:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762965.575151:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 29360128, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xe007 00010000:00010000:1.0:1789762965.575161:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05ef8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.575177:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:1.0:1789762965.575182:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762965.575188:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05ef8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.575203:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05ef8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762965.586125:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968296 is committed 00002000:00010000:3.0:1789762965.586140:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 29360128 -> 30408704 00002000:00010000:3.0:1789762965.586148:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 57351 -> 59399 00010000:00010000:3.0:1789762965.586159:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05ef8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.586170:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05ef8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.586176:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05ef8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.586186:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.586189:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05ef8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.586196:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05ef8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.586210:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05ef8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.586214:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05ef8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.586279:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81396680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015167360:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11213us (47523us total) trans 4294968296 rc 0/0 00000100:00100000:3.0:1789762965.586287:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1416 00000100:00100000:3.0:1789762965.586296:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488878879823 time 488878946040 state 0x2 ref 0 00000100:00100000:2.0:1789762965.588665:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762965.588699:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015167488 00000100:00100000:3.0:1789762965.588708:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015167488 00000100:00100000:3.0:1789762965.588747:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488878879823 time 488878946040 state 0x0 ref 1 00000100:00100000:3.0:1789762965.588758:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488928946040 time 488878946040 state 0x1 ref 1 00000100:00100000:2.0:1789762965.625224:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488928946040 time 488929122456 state 0x0 ref 1 00000100:00100000:2.0:1789762965.625246:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d81395180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015167488:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762965.625310:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 30408704, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xe807 00010000:00010000:2.0:1789762965.625319:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05eff lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.625334:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:2.0:1789762965.625338:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762965.625344:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05eff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.625363:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05eff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762965.634773:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967638 is committed 00000001:00080000:1.0:1789762965.634799:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967642 is committed 00000001:00080000:1.0:1789762965.634818:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967643 is committed 00000004:00080000:1.0:1789762965.634840:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 33 committed 00000004:00080000:1.0:1789762965.634852:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 26 committed 00000100:00100000:3.0:1789762965.634979:0:3272:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789762965.634988:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981d81394000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022201600:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789762965.634994:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022201728:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789762965.635049:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762965.635079:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022201600 00000100:00100000:3.0:1789762965.635106:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022201600 00000100:00100000:2.0:1789762965.635118:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762965.635179:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022201728 00000100:00100000:1.0:1789762965.635185:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022201728 00000100:00100000:3.0:1789762965.635185:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:3.0:1789762965.635200:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4970000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022201600:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1789762965.635222:0:6881:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:17 count 1 00000100:00100000:1.0:1789762965.635227:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:1.0:1789762965.635238:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022201728:12345-0@lo:6:osp-syn-0-0.0 00002000:00010000:3.0:1789762965.635250:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x11:0x0] initial LVB size: 8388608, mtime: 0x6aad9d93, atime: 0x0, ctime: 0x6aad9d93, blocks: 0x1 00002000:00080000:1.0:1789762965.635253:0:6648:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:26 count 1 00010000:00010000:3.0:1789762965.635261:0:6881:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b05f06 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.635273:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=488s v=(0 0) 00002000:00010000:1.0:1789762965.635278:0:6648:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1a:0x0] initial LVB size: 8388608, mtime: 0x6aad9d93, atime: 0x0, ctime: 0x6aad9d93, blocks: 0x4007 00010000:00010000:3.0:1789762965.635278:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762965.635293:0:6648:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05f0d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.635293:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b05f06 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.635317:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=488s v=(0 0) 00010000:00010000:1.0:1789762965.635321:0:6648:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.635323:0:6881:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b05f06 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.635325:0:6648:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05f0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.635344:0:6881:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b05f06 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.635346:0:6648:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05f0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.635365:0:6881:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b05f06 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.635367:0:6648:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05f0d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.635388:0:6648:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05f0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.635398:0:6881:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762965.635411:0:6648:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1789762965.635414:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1789762965.635423:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b05f06 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762965.635433:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762965.635438:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b05f06 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762965.635451:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b05f06 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.635453:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789762965.635462:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05f0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762965.635470:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b05f06 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.635473:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762965.635480:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05f0d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762965.635482:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000058de5a3/0x416193e012b05f06 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.635508:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05f0d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762965.635515:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762965.635519:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789762965.635522:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762965.635527:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05f0d lrc: 1/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.635538:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b05f0d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6648 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.635566:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762965.635567:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762965.635570:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762965.636357:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3272:x1876702022201728:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1121us (1241us total) trans 4294968298 rc 0/0 00000100:00100000:1.0:1789762965.636363:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:3.0:1789762965.636394:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4970000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022201600:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1197us (1343us total) trans 4294967451 rc 0/0 00000100:00100000:3.0:1789762965.636404:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000004:00080000:1.0:1789762965.636408:0:3272:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981ebfdfad80/2, rc 0, transno 4294968298 00000100:00100000:1.0:1789762965.636421:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022201728:0@lo:6:osp-syn-0-0.0 00000100:00080000:3.0:1789762965.636429:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789762965.636435:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000004:00080000:1.0:1789762965.636451:0:3274:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981d81394000/2, rc 0, transno 4294967451 00000100:00100000:1.0:1789762965.636465:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981d81394000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022201600:0@lo:6:osp-syn-1-0.0 00002000:00010000:2.0:1789762965.639257:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 30408704 -> 31457280 00000001:00080000:3.0:1789762965.639258:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968297 is committed 00002000:00010000:2.0:1789762965.639266:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 59399 -> 61447 00010000:00010000:2.0:1789762965.639285:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05eff lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.639296:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05eff lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.639303:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05eff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.639308:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762965.639310:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05eff lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.639315:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05eff lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.639324:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05eff lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.639337:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000416e0219/0x416193e012b05eff lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762965.639386:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d81395180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015167488:12345-192.168.203.48@tcp:4:dd.500 Request processed in 14148us (50720us total) trans 4294968297 rc 0/0 00000100:00100000:2.0:1789762965.639395:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1417 00000100:00100000:2.0:1789762965.639403:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488928946040 time 488929122456 state 0x2 ref 0 00000100:00100000:2.0:1789762965.641895:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762965.641937:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015167616 00000100:00100000:3.0:1789762965.641943:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015167616 00000100:00100000:3.0:1789762965.641972:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488928946040 time 488929122456 state 0x0 ref 1 00000100:00100000:3.0:1789762965.641982:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488979122456 time 488929122456 state 0x1 ref 1 00000100:00100000:3.0:1789762965.675281:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488979122456 time 488979182660 state 0x0 ref 1 00000100:00100000:3.0:1789762965.675298:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdcb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015167616:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762965.675344:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 31457280, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xf007 00010000:00010000:3.0:1789762965.675351:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f14 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.675363:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=488s v=(0 0) 00010000:00010000:3.0:1789762965.675367:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.675371:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.675381:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762965.687937:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968298 is committed 00000001:00080000:2.0:1789762965.687948:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968299 is committed 00002000:00010000:3.0:1789762965.687996:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 31457280 -> 32505856 00002000:00010000:3.0:1789762965.688000:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 61447 -> 63495 00010000:00010000:3.0:1789762965.688010:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f14 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.688020:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f14 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.688027:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.688032:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.688035:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f14 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.688042:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f14 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.688056:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f14 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.688061:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f14 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.688161:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdcb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015167616:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12864us (46262us total) trans 4294968299 rc 0/0 00000100:00100000:3.0:1789762965.688170:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1418 00000100:00100000:3.0:1789762965.688178:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488979122456 time 488979182660 state 0x2 ref 0 00000100:00100000:0.0:1789762965.690671:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762965.690751:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015167744 00000100:00100000:2.0:1789762965.690760:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015167744 00000100:00100000:2.0:1789762965.690799:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 488979122456 time 488979182660 state 0x0 ref 1 00000100:00100000:2.0:1789762965.690810:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489029182660 time 488979182660 state 0x1 ref 1 00000100:00100000:1.0:1789762965.725388:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489029182660 time 489029289171 state 0x0 ref 1 00000100:00100000:1.0:1789762965.725408:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015167744:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762965.725453:0:14913:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 32505856, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0xf807 00010000:00010000:1.0:1789762965.725460:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05f1b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.725474:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:1.0:1789762965.725477:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762965.725492:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05f1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.725505:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05f1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762965.736706:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968300 is committed 00002000:00010000:3.0:1789762965.736728:0:14913:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 32505856 -> 33554432 00002000:00010000:3.0:1789762965.736735:0:14913:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 63495 -> 65543 00010000:00010000:3.0:1789762965.736747:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05f1b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.736761:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05f1b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.736771:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05f1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.736779:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.736785:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05f1b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.736795:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05f1b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.736812:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05f1b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.736817:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b05f1b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.736899:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015167744:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11495us (46228us total) trans 4294968300 rc 0/0 00000100:00100000:3.0:1789762965.736908:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1419 00000100:00100000:3.0:1789762965.736917:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489029182660 time 489029289171 state 0x2 ref 0 00000100:00100000:2.0:1789762965.739502:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762965.739534:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015167872 00000100:00100000:3.0:1789762965.739541:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015167872 00000100:00100000:3.0:1789762965.739592:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489029182660 time 489029289171 state 0x0 ref 1 00000100:00100000:3.0:1789762965.739603:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489079289171 time 489029289171 state 0x1 ref 1 00000100:00100000:3.0:1789762965.775465:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489079289171 time 489079363237 state 0x0 ref 1 00000100:00100000:3.0:1789762965.775501:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015167872:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762965.775546:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 33554432, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0x10007 00010000:00010000:3.0:1789762965.775553:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f22 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.775566:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:3.0:1789762965.775570:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.775575:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.775585:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762965.786862:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968301 is committed 00002000:00010000:1.0:1789762965.786863:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 33554432 -> 34603008 00002000:00010000:1.0:1789762965.786870:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 65543 -> 67591 00010000:00010000:1.0:1789762965.786879:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f22 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.786890:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f22 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.786896:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.786902:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762965.786905:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f22 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.786912:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f22 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.786926:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f22 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762965.786931:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005b162c57/0x416193e012b05f22 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762965.786985:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc555180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015167872:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11504us (47483us total) trans 4294968301 rc 0/0 00000100:00100000:1.0:1789762965.787006:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1420 00000100:00100000:1.0:1789762965.787015:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489079289171 time 489079363237 state 0x2 ref 0 00000100:00100000:2.0:1789762965.789581:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762965.789634:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015168000 00000100:00100000:0.0:1789762965.789643:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015168000 00000100:00100000:0.0:1789762965.789681:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489079289171 time 489079363237 state 0x0 ref 1 00000100:00100000:0.0:1789762965.789692:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489129363237 time 489079363237 state 0x1 ref 1 00000100:00100000:3.0:1789762965.825538:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489129363237 time 489129440094 state 0x0 ref 1 00000100:00100000:3.0:1789762965.825556:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015168000:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762965.825591:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 34603008, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0x10807 00010000:00010000:3.0:1789762965.825597:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f29 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.825609:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:3.0:1789762965.825612:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.825616:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.825626:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762965.835744:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968302 is committed 00002000:00010000:3.0:1789762965.835803:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 34603008 -> 35651584 00002000:00010000:3.0:1789762965.835808:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 67591 -> 69639 00010000:00010000:3.0:1789762965.835816:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f29 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.835825:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f29 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.835832:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.835838:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.835842:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f29 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.835847:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f29 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.835861:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f29 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.835865:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f29 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.835915:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d846f2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015168000:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10363us (46334us total) trans 4294968302 rc 0/0 00000100:00100000:3.0:1789762965.835923:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1421 00000100:00100000:3.0:1789762965.835931:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489129363237 time 489129440094 state 0x2 ref 0 00000100:00100000:2.0:1789762965.838003:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762965.838083:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015168128 00000100:00100000:1.0:1789762965.838105:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015168128 00000100:00100000:1.0:1789762965.838154:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489129363237 time 489129440094 state 0x0 ref 1 00000100:00100000:1.0:1789762965.838165:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489179440094 time 489129440094 state 0x1 ref 1 00000100:00100000:2.0:1789762965.875637:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489179440094 time 489179537652 state 0x0 ref 1 00000100:00100000:2.0:1789762965.875660:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015168128:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762965.875701:0:17498:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 35651584, mtime: 0x6aad9d95, atime: 0x0, ctime: 0x6aad9d95, blocks: 0x11007 00010000:00010000:2.0:1789762965.875708:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4f84b4a/0x416193e012b05f30 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.875719:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:2.0:1789762965.875723:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762965.875728:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b4f84b4a/0x416193e012b05f30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762965.875751:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b4f84b4a/0x416193e012b05f30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762965.887717:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968303 is committed 00002000:00010000:3.0:1789762965.887783:0:17498:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 35651584 -> 36700160 00002000:00010000:3.0:1789762965.887787:0:17498:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb mtime from disk: 1789762965 -> 1789762966 00002000:00010000:3.0:1789762965.887789:0:17498:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb ctime from disk: 1789762965 -> 1789762966 00002000:00010000:3.0:1789762965.887791:0:17498:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 69639 -> 71687 00010000:00010000:3.0:1789762965.887800:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b4f84b4a/0x416193e012b05f30 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.887813:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b4f84b4a/0x416193e012b05f30 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.887820:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b4f84b4a/0x416193e012b05f30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.887824:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.887828:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4f84b4a/0x416193e012b05f30 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.887834:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b4f84b4a/0x416193e012b05f30 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.887846:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b4f84b4a/0x416193e012b05f30 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.887852:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b4f84b4a/0x416193e012b05f30 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.887906:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015168128:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12251us (49902us total) trans 4294968303 rc 0/0 00000100:00100000:3.0:1789762965.887914:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1422 00000100:00100000:3.0:1789762965.887922:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489179440094 time 489179537652 state 0x2 ref 0 00000100:00100000:0.0:1789762965.890074:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762965.890145:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015168256 00000100:00100000:3.0:1789762965.890152:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015168256 00000100:00100000:3.0:1789762965.890187:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489179440094 time 489179537652 state 0x0 ref 1 00000100:00100000:3.0:1789762965.890198:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489229537652 time 489179537652 state 0x1 ref 1 00000100:00100000:3.0:1789762965.925695:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489229537652 time 489229595556 state 0x0 ref 1 00000100:00100000:3.0:1789762965.925711:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfc1f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015168256:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762965.925744:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 36700160, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x11807 00010000:00010000:3.0:1789762965.925750:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f37 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.925761:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:3.0:1789762965.925765:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762965.925780:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.925789:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762965.936175:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968304 is committed 00002000:00010000:3.0:1789762965.936182:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 36700160 -> 37748736 00002000:00010000:3.0:1789762965.936188:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 71687 -> 73735 00010000:00010000:3.0:1789762965.936198:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f37 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.936207:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f37 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.936215:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.936220:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.936224:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f37 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.936230:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f37 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.936241:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f37 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.936246:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f37 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.936295:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfc1f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015168256:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10587us (46220us total) trans 4294968304 rc 0/0 00000100:00100000:3.0:1789762965.936303:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1423 00000100:00100000:3.0:1789762965.936312:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489229537652 time 489229595556 state 0x2 ref 0 00000100:00100000:2.0:1789762965.938472:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762965.938522:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015168384 00000100:00100000:1.0:1789762965.938530:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015168384 00000100:00100000:1.0:1789762965.938572:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489229537652 time 489229595556 state 0x0 ref 1 00000100:00100000:1.0:1789762965.938585:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489279595556 time 489229595556 state 0x1 ref 1 00000100:00100000:0.0:1789762965.975815:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489279595556 time 489279715482 state 0x0 ref 1 00000100:00100000:0.0:1789762965.975839:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015168384:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762965.975879:0:17497:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 37748736, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x12007 00010000:00010000:0.0:1789762965.975886:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b05f3e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762965.975909:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:0.0:1789762965.975913:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762965.975917:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b05f3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762965.975927:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b05f3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762965.987165:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968305 is committed 00002000:00010000:3.0:1789762965.987170:0:17497:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 37748736 -> 38797312 00002000:00010000:3.0:1789762965.987175:0:17497:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 73735 -> 75783 00010000:00010000:3.0:1789762965.987186:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b05f3e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.987200:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b05f3e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.987207:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b05f3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.987212:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762965.987216:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b05f3e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.987223:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b05f3e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.987237:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b05f3e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762965.987241:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008f6ca1d5/0x416193e012b05f3e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762965.987296:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015168384:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11463us (48823us total) trans 4294968305 rc 0/0 00000100:00100000:3.0:1789762965.987305:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1424 00000100:00100000:3.0:1789762965.987313:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489279595556 time 489279715482 state 0x2 ref 0 00000100:00100000:0.0:1789762965.989670:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762965.989713:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015168512 00000100:00100000:2.0:1789762965.989720:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015168512 00000100:00100000:2.0:1789762965.989757:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489279595556 time 489279715482 state 0x0 ref 1 00000100:00100000:2.0:1789762965.989768:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489329715482 time 489279715482 state 0x1 ref 1 00000100:00100000:1.0:1789762966.025886:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489329715482 time 489329787620 state 0x0 ref 1 00000100:00100000:1.0:1789762966.025902:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015168512:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762966.025938:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 38797312, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x12807 00010000:00010000:1.0:1789762966.025944:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f45 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.025956:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:1.0:1789762966.025959:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762966.025964:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.025974:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762966.036082:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 38797312 -> 39845888 00002000:00010000:1.0:1789762966.036102:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 75783 -> 77831 00000001:00080000:2.0:1789762966.036110:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968306 is committed 00010000:00010000:1.0:1789762966.036112:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f45 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.036123:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f45 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.036143:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.036149:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762966.036153:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f45 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.036160:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f45 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.036174:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f45 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.036183:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f45 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762966.036238:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015168512:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10338us (46567us total) trans 4294968306 rc 0/0 00000100:00100000:1.0:1789762966.036245:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1425 00000100:00100000:1.0:1789762966.036253:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489329715482 time 489329787620 state 0x2 ref 0 00000100:00100000:0.0:1789762966.038417:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762966.038457:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015168640 00000100:00100000:3.0:1789762966.038465:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015168640 00000100:00100000:3.0:1789762966.038500:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489329715482 time 489329787620 state 0x0 ref 1 00000100:00100000:3.0:1789762966.038511:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489379787620 time 489329787620 state 0x1 ref 1 00000100:00100000:3.0:1789762966.075958:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489379787620 time 489379858672 state 0x0 ref 1 00000100:00100000:3.0:1789762966.075977:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015168640:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762966.076017:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 39845888, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x13007 00010000:00010000:3.0:1789762966.076024:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f4c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.076036:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:3.0:1789762966.076039:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762966.076044:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.076054:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762966.086571:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968307 is committed 00002000:00010000:3.0:1789762966.086591:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 39845888 -> 40894464 00002000:00010000:3.0:1789762966.086597:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 77831 -> 79879 00010000:00010000:3.0:1789762966.086609:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f4c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.086622:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f4c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.086632:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.086637:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.086641:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f4c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.086647:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f4c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.086660:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f4c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.086666:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f4c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.086722:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015168640:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10749us (48304us total) trans 4294968307 rc 0/0 00000100:00100000:3.0:1789762966.086730:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1426 00000100:00100000:3.0:1789762966.086738:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489379787620 time 489379858672 state 0x2 ref 0 00000100:00100000:2.0:1789762966.088975:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762966.089058:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015168768 00000100:00100000:1.0:1789762966.089067:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015168768 00000100:00100000:1.0:1789762966.089141:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489379787620 time 489379858672 state 0x0 ref 1 00000100:00100000:1.0:1789762966.089153:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489429858672 time 489379858672 state 0x1 ref 1 00000100:00100000:1.0:1789762966.126023:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489429858672 time 489429921578 state 0x0 ref 1 00000100:00100000:1.0:1789762966.126040:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015168768:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762966.126080:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 40894464, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x13807 00010000:00010000:1.0:1789762966.126109:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f53 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.126122:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:1.0:1789762966.126125:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762966.126149:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.126160:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762966.136731:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968308 is committed 00002000:00010000:3.0:1789762966.136749:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 40894464 -> 41943040 00002000:00010000:3.0:1789762966.136754:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 79879 -> 81927 00010000:00010000:3.0:1789762966.136766:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f53 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.136785:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f53 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.136797:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.136805:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.136811:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f53 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.136819:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f53 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.136834:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f53 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.136839:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b05f53 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.136898:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015168768:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10860us (47921us total) trans 4294968308 rc 0/0 00000100:00100000:3.0:1789762966.136906:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1427 00000100:00100000:3.0:1789762966.136914:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489429858672 time 489429921578 state 0x2 ref 0 00000100:00100000:2.0:1789762966.139177:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762966.139211:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015168896 00000100:00100000:3.0:1789762966.139219:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015168896 00000100:00100000:3.0:1789762966.139263:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489429858672 time 489429921578 state 0x0 ref 1 00000100:00100000:3.0:1789762966.139274:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489479921578 time 489429921578 state 0x1 ref 1 00000100:00100000:2.0:1789762966.176143:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489479921578 time 489480030767 state 0x0 ref 1 00000100:00100000:2.0:1789762966.176162:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015168896:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762966.176200:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 41943040, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x14007 00010000:00010000:2.0:1789762966.176207:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d267301e/0x416193e012b05f5a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.176220:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:2.0:1789762966.176223:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762966.176228:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d267301e/0x416193e012b05f5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.176248:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d267301e/0x416193e012b05f5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762966.186904:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968309 is committed 00002000:00010000:3.0:1789762966.186921:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 41943040 -> 42991616 00002000:00010000:3.0:1789762966.186926:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 81927 -> 83975 00010000:00010000:3.0:1789762966.186935:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d267301e/0x416193e012b05f5a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.186945:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d267301e/0x416193e012b05f5a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.186952:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d267301e/0x416193e012b05f5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.186957:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.186960:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d267301e/0x416193e012b05f5a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.186967:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d267301e/0x416193e012b05f5a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.186981:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d267301e/0x416193e012b05f5a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.186986:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d267301e/0x416193e012b05f5a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.187037:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015168896:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10881us (47860us total) trans 4294968309 rc 0/0 00000100:00100000:3.0:1789762966.187045:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1428 00000100:00100000:3.0:1789762966.187054:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489479921578 time 489480030767 state 0x2 ref 0 00000100:00100000:0.0:1789762966.189360:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762966.189400:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015169024 00000100:00100000:3.0:1789762966.189407:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015169024 00000100:00100000:3.0:1789762966.189440:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489479921578 time 489480030767 state 0x0 ref 1 00000100:00100000:3.0:1789762966.189453:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489530030767 time 489480030767 state 0x1 ref 1 00000100:00100000:3.0:1789762966.226183:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489530030767 time 489530085195 state 0x0 ref 1 00000100:00100000:3.0:1789762966.226199:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015169024:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762966.226233:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 42991616, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x14807 00010000:00010000:3.0:1789762966.226239:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f61 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.226250:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:3.0:1789762966.226253:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762966.226258:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.226268:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762966.237027:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968310 is committed 00002000:00010000:3.0:1789762966.237046:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 42991616 -> 44040192 00002000:00010000:3.0:1789762966.237053:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 83975 -> 86023 00010000:00010000:3.0:1789762966.237064:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f61 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.237073:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f61 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.237080:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.237099:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.237103:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f61 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.237110:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f61 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.237123:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f61 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.237145:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f61 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.237202:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015169024:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11006us (47842us total) trans 4294968310 rc 0/0 00000100:00100000:3.0:1789762966.237211:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1429 00000100:00100000:3.0:1789762966.237220:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489530030767 time 489530085195 state 0x2 ref 0 00000100:00100000:0.0:1789762966.239404:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762966.239449:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015169152 00000100:00100000:2.0:1789762966.239457:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015169152 00000100:00100000:2.0:1789762966.239490:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489530030767 time 489530085195 state 0x0 ref 1 00000100:00100000:2.0:1789762966.239501:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489580085195 time 489530085195 state 0x1 ref 1 00000100:00100000:3.0:1789762966.276265:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489580085195 time 489580165221 state 0x0 ref 1 00000100:00100000:3.0:1789762966.276283:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015169152:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762966.276321:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 44040192, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x15007 00010000:00010000:3.0:1789762966.276327:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f68 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.276340:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:3.0:1789762966.276343:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762966.276348:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.276358:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762966.286619:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968311 is committed 00002000:00010000:3.0:1789762966.286634:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 44040192 -> 45088768 00002000:00010000:3.0:1789762966.286639:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 86023 -> 88073 00010000:00010000:3.0:1789762966.286648:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f68 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.286657:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f68 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.286669:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.286674:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.286678:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f68 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.286684:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f68 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.286697:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f68 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.286702:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f68 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.286752:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015169152:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10474us (47350us total) trans 4294968311 rc 0/0 00000100:00100000:3.0:1789762966.286760:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1430 00000100:00100000:3.0:1789762966.286768:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489580085195 time 489580165221 state 0x2 ref 0 00000100:00100000:3.0:1789762966.288913:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762966.288961:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015169280 00000100:00100000:1.0:1789762966.288968:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015169280 00000100:00100000:1.0:1789762966.289004:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489580085195 time 489580165221 state 0x0 ref 1 00000100:00100000:1.0:1789762966.289015:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489630165221 time 489580165221 state 0x1 ref 1 00000100:00100000:3.0:1789762966.326339:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489630165221 time 489630239345 state 0x0 ref 1 00000100:00100000:3.0:1789762966.326357:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015169280:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762966.326394:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 45088768, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x15809 00010000:00010000:3.0:1789762966.326401:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f6f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.326412:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:3.0:1789762966.326415:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762966.326420:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.326430:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762966.337791:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968312 is committed 00002000:00010000:3.0:1789762966.337816:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 45088768 -> 46137344 00002000:00010000:3.0:1789762966.337821:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 88073 -> 90121 00010000:00010000:3.0:1789762966.337830:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f6f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.337839:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f6f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.337845:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.337851:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.337854:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f6f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.337866:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f6f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.337880:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f6f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.337885:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f6f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.337939:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdf8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015169280:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11586us (49025us total) trans 4294968312 rc 0/0 00000100:00100000:3.0:1789762966.337948:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1431 00000100:00100000:3.0:1789762966.337956:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489630165221 time 489630239345 state 0x2 ref 0 00000100:00100000:2.0:1789762966.340357:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762966.340437:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015169408 00000100:00100000:0.0:1789762966.340444:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015169408 00000100:00100000:0.0:1789762966.340477:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489630165221 time 489630239345 state 0x0 ref 1 00000100:00100000:0.0:1789762966.340488:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489680239345 time 489630239345 state 0x1 ref 1 00000100:00100000:3.0:1789762966.376408:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489680239345 time 489680307716 state 0x0 ref 1 00000100:00100000:3.0:1789762966.376427:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015169408:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762966.376472:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 46137344, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x16009 00010000:00010000:3.0:1789762966.376482:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f76 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.376498:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:3.0:1789762966.376502:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762966.376507:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.376520:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762966.386915:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968313 is committed 00002000:00010000:3.0:1789762966.386915:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 46137344 -> 47185920 00002000:00010000:3.0:1789762966.386921:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 90121 -> 92169 00010000:00010000:3.0:1789762966.386931:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f76 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.386943:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f76 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.386954:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.386963:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.386968:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f76 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.386979:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f76 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.386996:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f76 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.387002:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f76 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.387056:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdff800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015169408:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10635us (46700us total) trans 4294968313 rc 0/0 00000100:00100000:3.0:1789762966.387065:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1432 00000100:00100000:3.0:1789762966.387074:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489680239345 time 489680307716 state 0x2 ref 0 00000100:00100000:2.0:1789762966.389365:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762966.389414:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015169536 00000100:00100000:1.0:1789762966.389422:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015169536 00000100:00100000:1.0:1789762966.389462:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489680239345 time 489680307716 state 0x0 ref 1 00000100:00100000:1.0:1789762966.389476:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489730307716 time 489680307716 state 0x1 ref 1 00000100:00100000:2.0:1789762966.426497:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489730307716 time 489730394658 state 0x0 ref 1 00000100:00100000:2.0:1789762966.426518:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015169536:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762966.426558:0:17498:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 47185920, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x16809 00010000:00010000:2.0:1789762966.426567:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05f7d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.426582:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:2.0:1789762966.426586:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762966.426592:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05f7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.426605:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05f7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762966.437679:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968314 is committed 00002000:00010000:3.0:1789762966.437689:0:17498:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 47185920 -> 48234496 00002000:00010000:3.0:1789762966.437694:0:17498:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 92169 -> 94217 00010000:00010000:3.0:1789762966.437704:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05f7d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.437714:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05f7d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.437720:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05f7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.437725:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.437729:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05f7d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.437736:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05f7d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.437750:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05f7d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.437765:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1bb2b89/0x416193e012b05f7d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.437830:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdffb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015169536:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11316us (48466us total) trans 4294968314 rc 0/0 00000100:00100000:3.0:1789762966.437839:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1433 00000100:00100000:3.0:1789762966.437848:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489730307716 time 489730394658 state 0x2 ref 0 00000100:00100000:2.0:1789762966.440123:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762966.440162:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015169664 00000100:00100000:3.0:1789762966.440171:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015169664 00000100:00100000:3.0:1789762966.440208:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489730307716 time 489730394658 state 0x0 ref 1 00000100:00100000:3.0:1789762966.440218:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489780394658 time 489730394658 state 0x1 ref 1 00010000:00010000:0.0:1789762966.455210:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762966.476579:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489780394658 time 489780478734 state 0x0 ref 1 00000100:00100000:1.0:1789762966.476600:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015169664:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762966.476653:0:14913:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 48234496, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x17009 00010000:00010000:1.0:1789762966.476663:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05f84 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.476677:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:1.0:1789762966.476681:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762966.476686:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05f84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.476696:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05f84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762966.487441:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968315 is committed 00002000:00010000:3.0:1789762966.487461:0:14913:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 48234496 -> 49283072 00002000:00010000:3.0:1789762966.487466:0:14913:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 94217 -> 96265 00010000:00010000:3.0:1789762966.487477:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05f84 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.487486:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05f84 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.487493:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05f84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.487503:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.487507:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05f84 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.487513:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05f84 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.487526:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05f84 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.487531:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b05f84 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.487588:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015169664:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10993us (47466us total) trans 4294968315 rc 0/0 00000100:00100000:3.0:1789762966.487597:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1434 00000100:00100000:3.0:1789762966.487606:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489780394658 time 489780478734 state 0x2 ref 0 00000100:00100000:2.0:1789762966.489819:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762966.489855:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015169792 00000100:00100000:3.0:1789762966.489862:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015169792 00000100:00100000:3.0:1789762966.489899:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489780394658 time 489780478734 state 0x0 ref 1 00000100:00100000:3.0:1789762966.489910:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489830478734 time 489780478734 state 0x1 ref 1 00000100:00100000:0.0:1789762966.526684:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489830478734 time 489830584383 state 0x0 ref 1 00000100:00100000:0.0:1789762966.526703:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015169792:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762966.526740:0:17497:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 49283072, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x17809 00010000:00010000:0.0:1789762966.526747:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b05f8b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762966.526758:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:0.0:1789762966.526762:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762966.526766:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b05f8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762966.526777:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b05f8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762966.538025:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968316 is committed 00002000:00010000:3.0:1789762966.538045:0:17497:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 49283072 -> 50331648 00002000:00010000:3.0:1789762966.538051:0:17497:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 96265 -> 98313 00010000:00010000:3.0:1789762966.538062:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b05f8b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.538075:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b05f8b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.538102:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b05f8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.538111:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.538115:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b05f8b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.538123:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b05f8b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.538152:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b05f8b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.538158:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000062a69944/0x416193e012b05f8b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.538220:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015169792:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11518us (48398us total) trans 4294968316 rc 0/0 00000100:00100000:3.0:1789762966.538227:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1435 00000100:00100000:3.0:1789762966.538235:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489830478734 time 489830584383 state 0x2 ref 0 00000100:00100000:0.0:1789762966.540347:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762966.540381:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015169920 00000100:00100000:3.0:1789762966.540389:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015169920 00000100:00100000:3.0:1789762966.540421:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489830478734 time 489830584383 state 0x0 ref 1 00000100:00100000:3.0:1789762966.540432:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489880584383 time 489830584383 state 0x1 ref 1 00000100:00100000:1.0:1789762966.576760:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489880584383 time 489880657539 state 0x0 ref 1 00000100:00100000:1.0:1789762966.576782:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015169920:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762966.576828:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 50331648, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x18009 00010000:00010000:1.0:1789762966.576836:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05f92 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.576850:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:1.0:1789762966.576855:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762966.576863:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05f92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.576877:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05f92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762966.589327:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968317 is committed 00002000:00010000:3.0:1789762966.589355:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 50331648 -> 51380224 00002000:00010000:3.0:1789762966.589361:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 98313 -> 100361 00010000:00010000:3.0:1789762966.589370:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05f92 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.589380:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05f92 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.589386:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05f92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.589391:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.589395:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05f92 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.589401:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05f92 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.589414:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05f92 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.589424:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b05f92 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.589480:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015169920:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12703us (49132us total) trans 4294968317 rc 0/0 00000100:00100000:3.0:1789762966.589488:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1436 00000100:00100000:3.0:1789762966.589496:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489880584383 time 489880657539 state 0x2 ref 0 00000100:00100000:3.0:1789762966.591842:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762966.591894:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015170048 00000100:00100000:2.0:1789762966.591903:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015170048 00000100:00100000:2.0:1789762966.591946:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489880584383 time 489880657539 state 0x0 ref 1 00000100:00100000:2.0:1789762966.591961:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489930657539 time 489880657539 state 0x1 ref 1 00000100:00100000:2.0:1789762966.626329:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762966.626376:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762966.626393:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015170176 00000100:00100000:0.0:1789762966.626402:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015170176 00000100:00100000:2.0:1789762966.626414:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762966.626446:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015170304 00000100:00100000:1.0:1789762966.626453:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015170304 00000100:00100000:0.0:1789762966.626455:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 364 00000100:00100000:2.0:1789762966.626461:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762966.626461:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015170432 00000100:00100000:3.0:1789762966.626469:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015170432 00000100:00100000:0.0:1789762966.626471:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85085050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015170176:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762966.626502:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015170560 00000100:00100000:2.0:1789762966.626510:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015170560 00000100:00100000:3.0:1789762966.626526:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 196 00000100:00100000:1.0:1789762966.626533:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 397 00000100:00100000:0.0:1789762966.626534:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85085050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015170176:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 64us (207us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762966.626543:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2366:x1876702015170432:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:0.0:1789762966.626549:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 364 00000100:00100000:1.0:1789762966.626553:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702015170304:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762966.626567:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 398 00000100:00100000:2.0:1789762966.626581:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015170560:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762966.626582:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+35:2366:x1876702015170432:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 44us (169us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762966.626595:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 196 00000100:00100000:1.0:1789762966.626607:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:2365:x1876702015170304:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 59us (230us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762966.626617:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 397 00000100:00100000:2.0:1789762966.626630:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015170560:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 51us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762966.626641:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 398 00000100:00080000:3.0:1789762966.626647:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789762966.626655:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762966.626810:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489930657539 time 489930713100 state 0x0 ref 1 00000100:00100000:3.0:1789762966.626821:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015170048:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762966.626856:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 51380224, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x18809 00010000:00010000:3.0:1789762966.626863:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f99 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.626877:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:3.0:1789762966.626880:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762966.626885:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.626895:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762966.639105:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 51380224 -> 52428800 00002000:00010000:3.0:1789762966.639110:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 100361 -> 102409 00010000:00010000:3.0:1789762966.639120:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f99 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762966.639121:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968318 is committed 00010000:00010000:3.0:1789762966.639146:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f99 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.639153:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.639159:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.639163:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f99 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.639170:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f99 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.639190:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f99 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.639196:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05f99 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.639246:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015170048:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12425us (47404us total) trans 4294968318 rc 0/0 00000100:00100000:3.0:1789762966.639254:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1437 00000100:00100000:3.0:1789762966.639262:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489930657539 time 489930713100 state 0x2 ref 0 00000100:00100000:2.0:1789762966.641715:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762966.641765:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015170688 00000100:00100000:0.0:1789762966.641772:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015170688 00000100:00100000:0.0:1789762966.641804:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489930657539 time 489930713100 state 0x0 ref 1 00000100:00100000:0.0:1789762966.641815:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489980713100 time 489930713100 state 0x1 ref 1 00000004:00080000:2.0:1789762966.647234:0:6868:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 489951145280ns, next at 490951142945ns, rc = 0 00000100:00100000:1.0:1789762966.647264:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfdfca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022201856:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762966.647309:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762966.647345:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022201856 00000100:00100000:0.0:1789762966.647351:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022201856 00000100:00100000:0.0:1789762966.647388:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:0.0:1789762966.647396:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022201856:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762966.647411:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 525795328 pending 0 free 3831496704 avail 3788505088 00000100:00100000:0.0:1789762966.647448:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3272:x1876702022201856:12345-0@lo:13:osp-pre-0-0.0 Request processed in 53us (139us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762966.647453:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:1.0:1789762966.647493:0:3272:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294968298 gen 2 00000004:00080000:1.0:1789762966.647501:0:3272:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981ebfdfad80, transno 4294968298 00000100:00100000:1.0:1789762966.647524:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfdfca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022201856:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762966.676879:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489980713100 time 489980781124 state 0x0 ref 1 00000100:00100000:3.0:1789762966.676896:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015170688:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762966.676929:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 52428800, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x19009 00010000:00010000:3.0:1789762966.676936:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.676949:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=489s v=(0 0) 00010000:00010000:3.0:1789762966.676952:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762966.676957:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.676967:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762966.686883:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968319 is committed 00002000:00010000:3.0:1789762966.686897:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 52428800 -> 53477376 00002000:00010000:3.0:1789762966.686902:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 102409 -> 104457 00010000:00010000:3.0:1789762966.686912:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.686921:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.686927:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.686932:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.686936:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.686943:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.686956:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.686961:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.687013:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015170688:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10120us (45297us total) trans 4294968319 rc 0/0 00000100:00100000:3.0:1789762966.687022:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1438 00000100:00100000:3.0:1789762966.687030:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489980713100 time 489980781124 state 0x2 ref 0 00000100:00100000:0.0:1789762966.689397:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762966.689453:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015170816 00000100:00100000:1.0:1789762966.689461:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015170816 00000100:00100000:1.0:1789762966.689500:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 489980713100 time 489980781124 state 0x0 ref 1 00000100:00100000:1.0:1789762966.689515:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490030781124 time 489980781124 state 0x1 ref 1 00000100:00100000:3.0:1789762966.726989:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490030781124 time 490030890807 state 0x0 ref 1 00000100:00100000:3.0:1789762966.727007:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015170816:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762966.727041:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 53477376, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x19809 00010000:00010000:3.0:1789762966.727048:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.727058:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:3.0:1789762966.727062:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762966.727066:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.727078:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762966.737473:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968320 is committed 00002000:00010000:3.0:1789762966.737524:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 53477376 -> 54525952 00002000:00010000:3.0:1789762966.737529:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 104457 -> 106505 00010000:00010000:3.0:1789762966.737537:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.737546:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.737553:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.737558:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.737562:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.737567:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.737580:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.737585:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fa7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.737636:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015170816:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10632us (48239us total) trans 4294968320 rc 0/0 00000100:00100000:3.0:1789762966.737648:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1439 00000100:00100000:3.0:1789762966.737659:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490030781124 time 490030890807 state 0x2 ref 0 00000100:00100000:2.0:1789762966.740072:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762966.740161:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015170944 00000100:00100000:0.0:1789762966.740168:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015170944 00000100:00100000:0.0:1789762966.740201:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490030781124 time 490030890807 state 0x0 ref 1 00000100:00100000:0.0:1789762966.740211:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490080890807 time 490030890807 state 0x1 ref 1 00000100:00100000:2.0:1789762966.777109:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490080890807 time 490080979351 state 0x0 ref 1 00000100:00100000:2.0:1789762966.777149:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015170944:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762966.777199:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 54525952, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x1a009 00010000:00010000:2.0:1789762966.777208:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05fae lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.777223:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:2.0:1789762966.777228:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762966.777234:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05fae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.777249:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05fae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762966.789471:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968321 is committed 00002000:00010000:2.0:1789762966.789473:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 54525952 -> 55574528 00002000:00010000:2.0:1789762966.789480:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 106505 -> 108553 00010000:00010000:2.0:1789762966.789492:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05fae lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.789504:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05fae lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.789514:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05fae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.789535:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762966.789540:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05fae lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.789551:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05fae lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.789570:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05fae lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762966.789579:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b05fae lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762966.789646:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015170944:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12506us (49572us total) trans 4294968321 rc 0/0 00000100:00100000:2.0:1789762966.789658:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1440 00000100:00100000:2.0:1789762966.789670:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490080890807 time 490080979351 state 0x2 ref 0 00000100:00100000:3.0:1789762966.792227:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762966.792321:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015171072 00000100:00100000:0.0:1789762966.792329:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015171072 00000100:00100000:0.0:1789762966.792368:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490080890807 time 490080979351 state 0x0 ref 1 00000100:00100000:0.0:1789762966.792383:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490130979351 time 490080979351 state 0x1 ref 1 00000100:00100000:0.0:1789762966.827155:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490130979351 time 490131052237 state 0x0 ref 1 00000100:00100000:0.0:1789762966.827178:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015171072:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762966.827229:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 55574528, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x1a809 00010000:00010000:0.0:1789762966.827243:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05fb5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762966.827256:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:0.0:1789762966.827260:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762966.827264:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05fb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762966.827275:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05fb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762966.838520:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968322 is committed 00002000:00010000:3.0:1789762966.838547:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 55574528 -> 56623104 00002000:00010000:3.0:1789762966.838554:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 108553 -> 110601 00010000:00010000:3.0:1789762966.838565:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05fb5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.838578:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05fb5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.838589:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05fb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.838597:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.838601:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05fb5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.838612:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05fb5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.838632:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05fb5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.838641:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000033d7c9c2/0x416193e012b05fb5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.838701:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015171072:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11530us (46474us total) trans 4294968322 rc 0/0 00000100:00100000:3.0:1789762966.838709:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1441 00000100:00100000:3.0:1789762966.838718:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490130979351 time 490131052237 state 0x2 ref 0 00000100:00100000:3.0:1789762966.841601:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762966.841653:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015171200 00000100:00100000:1.0:1789762966.841662:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015171200 00000100:00100000:1.0:1789762966.841706:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490130979351 time 490131052237 state 0x0 ref 1 00000100:00100000:1.0:1789762966.841722:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490181052237 time 490131052237 state 0x1 ref 1 00000100:00100000:1.0:1789762966.877216:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490181052237 time 490181111182 state 0x0 ref 1 00000100:00100000:1.0:1789762966.877243:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015171200:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762966.877295:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 56623104, mtime: 0x6aad9d96, atime: 0x0, ctime: 0x6aad9d96, blocks: 0x1b009 00010000:00010000:1.0:1789762966.877305:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05fbc lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.877321:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:1.0:1789762966.877326:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762966.877334:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05fbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762966.877348:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05fbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762966.888961:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968323 is committed 00002000:00010000:3.0:1789762966.889001:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 56623104 -> 57671680 00002000:00010000:3.0:1789762966.889006:0:6653:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb mtime from disk: 1789762966 -> 1789762967 00002000:00010000:3.0:1789762966.889008:0:6653:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb ctime from disk: 1789762966 -> 1789762967 00002000:00010000:3.0:1789762966.889010:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 110601 -> 112649 00010000:00010000:3.0:1789762966.889020:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05fbc lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.889029:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05fbc lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.889036:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05fbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.889041:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.889044:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05fbc lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.889051:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05fbc lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.889065:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05fbc lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.889071:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b05fbc lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.889181:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4bcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015171200:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11945us (47577us total) trans 4294968323 rc 0/0 00000100:00100000:3.0:1789762966.889194:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1442 00000100:00100000:3.0:1789762966.889207:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490181052237 time 490181111182 state 0x2 ref 0 00000100:00100000:3.0:1789762966.891938:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762966.891994:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015171328 00000100:00100000:1.0:1789762966.892003:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015171328 00000100:00100000:1.0:1789762966.892046:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490181052237 time 490181111182 state 0x0 ref 1 00000100:00100000:1.0:1789762966.892062:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490231111182 time 490181111182 state 0x1 ref 1 00000100:00100000:0.0:1789762966.927287:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490231111182 time 490231187815 state 0x0 ref 1 00000100:00100000:0.0:1789762966.927304:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015171328:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762966.927342:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 57671680, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x1b809 00010000:00010000:0.0:1789762966.927350:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b05fc3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762966.927361:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:0.0:1789762966.927365:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762966.927369:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b05fc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762966.927380:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b05fc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762966.939066:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968324 is committed 00002000:00010000:3.0:1789762966.939148:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 57671680 -> 58720256 00002000:00010000:3.0:1789762966.939154:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 112649 -> 114697 00010000:00010000:3.0:1789762966.939166:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b05fc3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.939179:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b05fc3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.939189:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b05fc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.939198:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.939203:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b05fc3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.939213:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b05fc3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.939229:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b05fc3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.939235:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000754ce020/0x416193e012b05fc3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.939299:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015171328:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11998us (47360us total) trans 4294968324 rc 0/0 00000100:00100000:3.0:1789762966.939307:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1443 00000100:00100000:3.0:1789762966.939315:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490231111182 time 490231187815 state 0x2 ref 0 00000100:00100000:2.0:1789762966.942246:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762966.942281:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015171456 00000100:00100000:3.0:1789762966.942290:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015171456 00000100:00100000:3.0:1789762966.942326:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490231111182 time 490231187815 state 0x0 ref 1 00000100:00100000:3.0:1789762966.942337:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490281187815 time 490231187815 state 0x1 ref 1 00000100:00100000:0.0:1789762966.977389:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490281187815 time 490281280874 state 0x0 ref 1 00000100:00100000:0.0:1789762966.977421:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015171456:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762966.977479:0:17498:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 58720256, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x1c009 00010000:00010000:0.0:1789762966.977489:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05fca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762966.977507:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:0.0:1789762966.977513:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762966.977521:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05fca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762966.977537:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05fca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762966.988550:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968325 is committed 00002000:00010000:3.0:1789762966.988569:0:17498:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 58720256 -> 59768832 00002000:00010000:3.0:1789762966.988574:0:17498:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 114697 -> 116745 00010000:00010000:3.0:1789762966.988585:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05fca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.988594:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05fca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.988601:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05fca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.988606:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d2400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762966.988609:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05fca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.988616:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05fca lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.988631:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05fca lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762966.988636:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ee0d6ee3/0x416193e012b05fca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762966.988695:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdff480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015171456:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11287us (46448us total) trans 4294968325 rc 0/0 00000100:00100000:3.0:1789762966.988703:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1444 00000100:00100000:3.0:1789762966.988713:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490281187815 time 490281280874 state 0x2 ref 0 00000100:00100000:0.0:1789762966.991055:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762966.991106:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015171584 00000100:00100000:3.0:1789762966.991115:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015171584 00000100:00100000:3.0:1789762966.991166:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490281187815 time 490281280874 state 0x0 ref 1 00000100:00100000:3.0:1789762966.991177:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490331280874 time 490281280874 state 0x1 ref 1 00000100:00100000:1.0:1789762967.027526:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490331280874 time 490331426258 state 0x0 ref 1 00000100:00100000:1.0:1789762967.027557:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015171584:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:3.0:1789762967.027580:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789762967.027598:0:14913:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 59768832, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x1c809 00010000:00010000:1.0:1789762967.027606:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b05fd1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.027619:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:1.0:1789762967.027622:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762967.027627:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b05fd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.027638:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b05fd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762967.038922:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968326 is committed 00002000:00010000:3.0:1789762967.038946:0:14913:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 59768832 -> 60817408 00002000:00010000:3.0:1789762967.038951:0:14913:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 116745 -> 118793 00010000:00010000:3.0:1789762967.038960:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b05fd1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.038975:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b05fd1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.038981:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b05fd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.038986:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.038990:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b05fd1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.038996:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b05fd1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.039010:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b05fd1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.039015:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b05fd1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.039071:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad02d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015171584:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11530us (48015us total) trans 4294968326 rc 0/0 00000100:00100000:3.0:1789762967.039080:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1445 00000100:00100000:3.0:1789762967.039114:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490331280874 time 490331426258 state 0x2 ref 0 00000100:00100000:0.0:1789762967.041351:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762967.041387:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015171712 00000100:00100000:3.0:1789762967.041395:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015171712 00000100:00100000:3.0:1789762967.041431:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490331280874 time 490331426258 state 0x0 ref 1 00000100:00100000:3.0:1789762967.041442:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490381426258 time 490331426258 state 0x1 ref 1 00000100:00100000:3.0:1789762967.077590:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490381426258 time 490381486981 state 0x0 ref 1 00000100:00100000:3.0:1789762967.077611:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015171712:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.077661:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 60817408, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x1d009 00010000:00010000:3.0:1789762967.077671:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fd8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.077688:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:3.0:1789762967.077694:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.077700:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.077712:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762967.088864:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968327 is committed 00002000:00010000:3.0:1789762967.088890:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 60817408 -> 61865984 00002000:00010000:3.0:1789762967.088897:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 118793 -> 120841 00010000:00010000:3.0:1789762967.088910:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fd8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.088920:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fd8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.088927:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.088933:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.088936:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fd8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.088943:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fd8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.088957:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fd8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.088962:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fd8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.089023:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015171712:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11418us (47672us total) trans 4294968327 rc 0/0 00000100:00100000:3.0:1789762967.089032:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1446 00000100:00100000:3.0:1789762967.089041:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490381426258 time 490381486981 state 0x2 ref 0 00000100:00100000:0.0:1789762967.091540:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762967.091597:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015171840 00000100:00100000:1.0:1789762967.091608:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015171840 00000100:00100000:1.0:1789762967.091649:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490381426258 time 490381486981 state 0x0 ref 1 00000100:00100000:1.0:1789762967.091661:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490431486981 time 490381486981 state 0x1 ref 1 00000100:00100000:3.0:1789762967.127700:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490431486981 time 490431591723 state 0x0 ref 1 00000100:00100000:3.0:1789762967.127725:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015171840:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.127787:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 61865984, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x1d809 00010000:00010000:3.0:1789762967.127794:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fdf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.127809:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:3.0:1789762967.127813:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.127818:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.127829:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762967.140165:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968328 is committed 00002000:00010000:3.0:1789762967.140176:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 61865984 -> 62914560 00002000:00010000:3.0:1789762967.140182:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 120841 -> 122889 00010000:00010000:3.0:1789762967.140195:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fdf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.140209:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fdf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.140219:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fdf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.140227:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.140231:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fdf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.140244:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fdf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.140258:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fdf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.140263:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fdf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.140325:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015171840:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12608us (48784us total) trans 4294968328 rc 0/0 00000100:00100000:3.0:1789762967.140333:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1447 00000100:00100000:3.0:1789762967.140343:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490431486981 time 490431591723 state 0x2 ref 0 00000100:00100000:2.0:1789762967.142866:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762967.142901:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015171968 00000100:00100000:3.0:1789762967.142910:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015171968 00000100:00100000:3.0:1789762967.142946:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490431486981 time 490431591723 state 0x0 ref 1 00000100:00100000:3.0:1789762967.142958:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490481591723 time 490431591723 state 0x1 ref 1 00000004:00080000:3.0:1789762967.159234:0:7726:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 490463145583ns, next at 491463143118ns, rc = 0 00000100:00100000:1.0:1789762967.159254:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022202112:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.159273:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad02a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022202240:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.159286:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ea0fc7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022201984:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762967.159315:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762967.159315:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762967.159324:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad00e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022202368:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.159333:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762967.159366:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762967.159375:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad03800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022202496:0@lo:400:kworker.0 00000100:00100000:1.0:1789762967.159381:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebad01c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022202624:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.159381:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ead4f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022202752:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.159413:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762967.159420:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022202240 00000100:00100000:0.0:1789762967.159426:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022202240 00000100:00100000:1.0:1789762967.159428:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762967.159439:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022202368 00000100:00100000:3.0:1789762967.159445:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022202368 00000100:00100000:0.0:1789762967.159465:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022202624 00000100:00100000:0.0:1789762967.159469:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022202624 00000100:00100000:2.0:1789762967.159483:0:6650:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022201984 00000100:00100000:1.0:1789762967.159486:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022202112 00000100:00100000:2.0:1789762967.159490:0:6650:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022201984 00000100:00100000:1.0:1789762967.159493:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022202112 00000100:00100000:3.0:1789762967.159497:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:0.0:1789762967.159507:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:3.0:1789762967.159511:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022202240:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762967.159520:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022202368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.159535:0:6650:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:2.0:1789762967.159546:0:6650:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022201984:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762967.159553:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:1.0:1789762967.159564:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85087850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022202112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.159568:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3271:x1876702022202240:12345-0@lo:400:kworker.0 Request processed in 63us (255us total) trans 0 rc 0/0 00002000:00000024:2.0:1789762967.159573:0:6650:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 97058816 pending 25559040 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789762967.159577:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3272:x1876702022202368:12345-0@lo:400:kworker.0 Request processed in 60us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762967.159580:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:0.0:1789762967.159590:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:0.0:1789762967.159609:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1789762967.159618:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4970700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022202624:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789762967.159622:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789762967.159630:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762967.159633:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85087850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022202112:12345-0@lo:400:kworker.0 Request processed in 71us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762967.159637:0:6650:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc868000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022201984:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (302us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762967.159653:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00100000:2.0:1789762967.159655:0:6650:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:0.0:1789762967.159656:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4970700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022202624:12345-0@lo:400:kworker.0 Request processed in 36us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762967.159661:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022202496 00000100:00100000:0.0:1789762967.159668:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:3.0:1789762967.159668:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022202496 00000100:00100000:0.0:1789762967.159714:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad00e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3272:1876702022202368:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.159718:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ea0fc7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022201984:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789762967.159718:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:3.0:1789762967.159735:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022202496:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762967.159754:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762967.159759:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022202624:0@lo:400:kworker.0 00000100:00080000:2.0:1789762967.159759:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789762967.159778:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad01500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022202112:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.159790:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022202752 00000100:00100000:2.0:1789762967.159795:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022202752 00000100:00100000:3.0:1789762967.159797:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022202496:12345-0@lo:400:kworker.0 Request processed in 65us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762967.159799:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad02a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3271:1876702022202240:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.159813:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:2.0:1789762967.159829:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:2.0:1789762967.159839:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc436a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022202752:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.159872:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebad03800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022202496:0@lo:400:kworker.0 00000100:00100000:2.0:1789762967.159879:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc436a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022202752:12345-0@lo:400:kworker.0 Request processed in 42us (125us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762967.159885:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:0.0:1789762967.159911:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ead4f7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022202752:0@lo:400:kworker.0 00000100:00100000:3.0:1789762967.177733:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490481591723 time 490481637099 state 0x0 ref 1 00000100:00100000:3.0:1789762967.177746:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfdfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015171968:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.177777:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 62914560, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x1e009 00010000:00010000:3.0:1789762967.177784:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fe6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.177796:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:3.0:1789762967.177800:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.177804:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fe6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.177815:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fe6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762967.188440:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968329 is committed 00002000:00010000:3.0:1789762967.188455:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 62914560 -> 63963136 00002000:00010000:3.0:1789762967.188460:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 122889 -> 124937 00010000:00010000:3.0:1789762967.188469:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fe6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.188478:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fe6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.188485:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fe6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.188490:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.188493:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fe6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.188502:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fe6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.188521:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fe6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.188538:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fe6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.188592:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfdfca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015171968:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10847us (45728us total) trans 4294968329 rc 0/0 00000100:00100000:3.0:1789762967.188600:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1448 00000100:00100000:3.0:1789762967.188608:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490481591723 time 490481637099 state 0x2 ref 0 00000100:00100000:2.0:1789762967.190808:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762967.190849:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015172096 00000100:00100000:0.0:1789762967.190857:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015172096 00000100:00100000:0.0:1789762967.190887:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490481591723 time 490481637099 state 0x0 ref 1 00000100:00100000:0.0:1789762967.190898:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490531637099 time 490481637099 state 0x1 ref 1 00000100:00100000:3.0:1789762967.227803:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490531637099 time 490531704200 state 0x0 ref 1 00000100:00100000:3.0:1789762967.227820:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015172096:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.227855:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 63963136, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x1e809 00010000:00010000:3.0:1789762967.227861:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fed lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.227882:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:3.0:1789762967.227885:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.227890:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.227900:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762967.240350:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968330 is committed 00002000:00010000:3.0:1789762967.240371:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 63963136 -> 65011712 00002000:00010000:3.0:1789762967.240376:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 124937 -> 126985 00010000:00010000:3.0:1789762967.240387:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fed lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.240398:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fed lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.240404:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.240409:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.240413:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fed lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.240420:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fed lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.240435:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fed lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.240442:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05fed lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.240521:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015172096:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12702us (49709us total) trans 4294968330 rc 0/0 00000100:00100000:3.0:1789762967.240532:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1449 00000100:00100000:3.0:1789762967.240544:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490531637099 time 490531704200 state 0x2 ref 0 00000100:00100000:2.0:1789762967.242909:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762967.242948:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015172224 00000100:00100000:1.0:1789762967.242956:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015172224 00000100:00100000:1.0:1789762967.242995:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490531637099 time 490531704200 state 0x0 ref 1 00000100:00100000:1.0:1789762967.243006:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490581704200 time 490531704200 state 0x1 ref 1 00000100:00100000:1.0:1789762967.277886:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490581704200 time 490581782134 state 0x0 ref 1 00000100:00100000:1.0:1789762967.277911:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015172224:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762967.277960:0:17497:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 65011712, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x1f009 00010000:00010000:1.0:1789762967.277967:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05ff4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.277981:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:1.0:1789762967.277984:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762967.277989:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05ff4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.278000:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05ff4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762967.289575:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968331 is committed 00002000:00010000:1.0:1789762967.289576:0:17497:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 65011712 -> 66060288 00002000:00010000:1.0:1789762967.289582:0:17497:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 126985 -> 129033 00010000:00010000:1.0:1789762967.289592:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05ff4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.289602:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05ff4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.289609:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05ff4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.289614:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762967.289618:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05ff4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.289624:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05ff4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.289639:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05ff4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.289644:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b05ff4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762967.289706:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015172224:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11803us (46796us total) trans 4294968331 rc 0/0 00000100:00100000:1.0:1789762967.289715:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1450 00000100:00100000:1.0:1789762967.289724:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490581704200 time 490581782134 state 0x2 ref 0 00000100:00100000:0.0:1789762967.292010:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762967.292053:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015172352 00000100:00100000:3.0:1789762967.292061:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015172352 00000100:00100000:3.0:1789762967.292115:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490581704200 time 490581782134 state 0x0 ref 1 00000100:00100000:3.0:1789762967.292138:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490631782134 time 490581782134 state 0x1 ref 1 00000100:00100000:3.0:1789762967.327973:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490631782134 time 490631863527 state 0x0 ref 1 00000100:00100000:3.0:1789762967.328004:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc435500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015172352:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.328060:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 66060288, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x1f809 00010000:00010000:3.0:1789762967.328070:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05ffb lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.328112:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:3.0:1789762967.328131:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.328138:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05ffb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.328154:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05ffb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762967.339274:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968332 is committed 00002000:00010000:3.0:1789762967.339337:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 66060288 -> 67108864 00002000:00010000:3.0:1789762967.339342:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 129033 -> 131081 00010000:00010000:3.0:1789762967.339353:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05ffb lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.339363:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05ffb lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.339370:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05ffb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.339375:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.339379:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05ffb lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.339385:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05ffb lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.339399:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05ffb lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.339404:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b05ffb lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.339470:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc435500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015172352:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11476us (47459us total) trans 4294968332 rc 0/0 00000100:00100000:3.0:1789762967.339482:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1451 00000100:00100000:3.0:1789762967.339493:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490631782134 time 490631863527 state 0x2 ref 0 00000100:00100000:0.0:1789762967.341999:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762967.342034:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015172480 00000100:00100000:3.0:1789762967.342042:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015172480 00000100:00100000:3.0:1789762967.342080:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490631782134 time 490631863527 state 0x0 ref 1 00000100:00100000:3.0:1789762967.342107:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490681863527 time 490631863527 state 0x1 ref 1 00000100:00100000:0.0:1789762967.378077:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490681863527 time 490681974664 state 0x0 ref 1 00000100:00100000:0.0:1789762967.378134:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015172480:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:0.0:1789762967.378176:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 67108864, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x20009 00010000:00010000:0.0:1789762967.378183:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b06002 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762967.378195:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:0.0:1789762967.378198:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762967.378203:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b06002 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762967.378213:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b06002 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762967.389270:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968333 is committed 00002000:00010000:3.0:1789762967.389284:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 67108864 -> 68157440 00002000:00010000:3.0:1789762967.389290:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 131081 -> 133129 00010000:00010000:3.0:1789762967.389300:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b06002 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.389311:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b06002 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.389317:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b06002 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.389323:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.389326:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b06002 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.389333:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b06002 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.389346:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b06002 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.389351:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c7ff4bb7/0x416193e012b06002 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.389408:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc437800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015172480:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11294us (47409us total) trans 4294968333 rc 0/0 00000100:00100000:3.0:1789762967.389417:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1452 00000100:00100000:3.0:1789762967.389426:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490681863527 time 490681974664 state 0x2 ref 0 00000100:00100000:2.0:1789762967.391899:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762967.391933:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015172608 00000100:00100000:3.0:1789762967.391940:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015172608 00000100:00100000:3.0:1789762967.391973:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490681863527 time 490681974664 state 0x0 ref 1 00000100:00100000:3.0:1789762967.391984:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490731974664 time 490681974664 state 0x1 ref 1 00000100:00100000:1.0:1789762967.428190:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490731974664 time 490732089810 state 0x0 ref 1 00000100:00100000:1.0:1789762967.428209:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015172608:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762967.428253:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 68157440, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x20809 00010000:00010000:1.0:1789762967.428260:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b06009 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.428272:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:1.0:1789762967.428276:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762967.428281:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b06009 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.428291:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b06009 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762967.439248:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968334 is committed 00002000:00010000:3.0:1789762967.439271:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 68157440 -> 69206016 00002000:00010000:3.0:1789762967.439276:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 133129 -> 135177 00010000:00010000:3.0:1789762967.439287:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b06009 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.439297:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b06009 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.439303:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b06009 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.439308:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.439312:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b06009 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.439320:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b06009 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.439335:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b06009 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.439340:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b06009 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.439398:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015172608:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11190us (47496us total) trans 4294968334 rc 0/0 00000100:00100000:3.0:1789762967.439410:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1453 00000100:00100000:3.0:1789762967.439423:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490731974664 time 490732089810 state 0x2 ref 0 00000100:00100000:0.0:1789762967.441952:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762967.441987:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015172736 00000100:00100000:3.0:1789762967.441995:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015172736 00000100:00100000:3.0:1789762967.442033:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490731974664 time 490732089810 state 0x0 ref 1 00000100:00100000:3.0:1789762967.442047:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490782089810 time 490732089810 state 0x1 ref 1 00000100:00100000:3.0:1789762967.478266:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490782089810 time 490782164031 state 0x0 ref 1 00000100:00100000:3.0:1789762967.478286:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc434e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015172736:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.478332:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 69206016, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x21009 00010000:00010000:3.0:1789762967.478339:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06010 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.478352:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:3.0:1789762967.478356:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.478360:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06010 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.478371:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06010 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.479234:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00002000:00010000:3.0:1789762967.489331:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 69206016 -> 70254592 00000001:00080000:1.0:1789762967.489336:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968335 is committed 00002000:00010000:3.0:1789762967.489337:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 135177 -> 137225 00010000:00010000:3.0:1789762967.489351:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06010 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.489361:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06010 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.489367:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06010 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.489373:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.489376:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06010 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.489383:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06010 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.489398:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06010 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.489404:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06010 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.489466:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc434e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015172736:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11183us (47512us total) trans 4294968335 rc 0/0 00000100:00100000:3.0:1789762967.489473:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1454 00000100:00100000:3.0:1789762967.489482:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490782089810 time 490782164031 state 0x2 ref 0 00000100:00100000:0.0:1789762967.492032:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762967.492110:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015172864 00000100:00100000:1.0:1789762967.492133:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015172864 00000100:00100000:1.0:1789762967.492191:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490782089810 time 490782164031 state 0x0 ref 1 00000100:00100000:1.0:1789762967.492207:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490832164031 time 490782164031 state 0x1 ref 1 00000100:00100000:3.0:1789762967.528360:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490832164031 time 490832258444 state 0x0 ref 1 00000100:00100000:3.0:1789762967.528380:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ead4f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015172864:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.528431:0:17498:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 70254592, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x21809 00010000:00010000:3.0:1789762967.528441:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06017 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.528458:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:3.0:1789762967.528462:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.528467:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06017 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.528478:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06017 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762967.539316:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968336 is committed 00002000:00010000:3.0:1789762967.539325:0:17498:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 70254592 -> 71303168 00002000:00010000:3.0:1789762967.539331:0:17498:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 137225 -> 139273 00010000:00010000:3.0:1789762967.539340:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06017 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.539350:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06017 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.539357:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06017 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.539362:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.539366:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06017 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.539374:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06017 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.539386:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06017 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.539392:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06017 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.539471:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ead4f7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015172864:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11095us (47438us total) trans 4294968336 rc 0/0 00000100:00100000:3.0:1789762967.539479:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1455 00000100:00100000:3.0:1789762967.539487:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490832164031 time 490832258444 state 0x2 ref 0 00000100:00100000:0.0:1789762967.541752:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762967.541799:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015172992 00000100:00100000:2.0:1789762967.541807:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015172992 00000100:00100000:2.0:1789762967.541844:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490832164031 time 490832258444 state 0x0 ref 1 00000100:00100000:2.0:1789762967.541855:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490882258444 time 490832258444 state 0x1 ref 1 00000100:00100000:3.0:1789762967.578463:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490882258444 time 490882360963 state 0x0 ref 1 00000100:00100000:3.0:1789762967.578496:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015172992:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.578547:0:14913:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 71303168, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x22009 00010000:00010000:3.0:1789762967.578555:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0601e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.578567:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:3.0:1789762967.578570:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.578575:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0601e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.578585:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0601e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762967.589801:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968337 is committed 00002000:00010000:3.0:1789762967.589857:0:14913:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 71303168 -> 72351744 00002000:00010000:3.0:1789762967.589862:0:14913:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 139273 -> 141321 00010000:00010000:3.0:1789762967.589872:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0601e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.589881:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0601e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.589888:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0601e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.589894:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.589897:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0601e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.589903:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0601e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.589916:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0601e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.589921:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0601e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.589992:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad03800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015172992:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11514us (48239us total) trans 4294968337 rc 0/0 00000100:00100000:3.0:1789762967.590000:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1456 00000100:00100000:3.0:1789762967.590008:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490882258444 time 490882360963 state 0x2 ref 0 00000100:00100000:2.0:1789762967.592405:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762967.592470:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015173120 00000100:00100000:1.0:1789762967.592480:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015173120 00000100:00100000:1.0:1789762967.592528:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490882258444 time 490882360963 state 0x0 ref 1 00000100:00100000:1.0:1789762967.592545:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490932360963 time 490882360963 state 0x1 ref 1 00000100:00100000:3.0:1789762967.628558:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490932360963 time 490932457304 state 0x0 ref 1 00000100:00100000:3.0:1789762967.628587:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015173120:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.628643:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 72351744, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x22809 00010000:00010000:3.0:1789762967.628650:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06025 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.628664:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:3.0:1789762967.628667:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.628672:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06025 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.628683:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06025 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762967.640249:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968338 is committed 00002000:00010000:3.0:1789762967.640270:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 72351744 -> 73400320 00002000:00010000:3.0:1789762967.640275:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 141321 -> 143369 00010000:00010000:3.0:1789762967.640286:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06025 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.640295:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06025 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.640302:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06025 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.640308:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.640311:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06025 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.640322:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06025 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.640336:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06025 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.640341:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06025 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.640413:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015173120:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11832us (48006us total) trans 4294968338 rc 0/0 00000100:00100000:3.0:1789762967.640422:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1457 00000100:00100000:3.0:1789762967.640431:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490932360963 time 490932457304 state 0x2 ref 0 00000100:00100000:0.0:1789762967.642927:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762967.643005:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015173248 00000100:00100000:1.0:1789762967.643014:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015173248 00000100:00100000:1.0:1789762967.643053:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490932360963 time 490932457304 state 0x0 ref 1 00000100:00100000:1.0:1789762967.643075:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490982457304 time 490932457304 state 0x1 ref 1 00000100:00100000:3.0:1789762967.678681:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490982457304 time 490982576825 state 0x0 ref 1 00000100:00100000:3.0:1789762967.678706:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015173248:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.678765:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 73400320, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x23009 00010000:00010000:3.0:1789762967.678772:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0602c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.678786:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=490s v=(0 0) 00010000:00010000:3.0:1789762967.678790:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.678794:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0602c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.678806:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0602c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762967.689829:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968339 is committed 00002000:00010000:3.0:1789762967.689841:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 73400320 -> 74448896 00002000:00010000:3.0:1789762967.689847:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 143369 -> 145417 00010000:00010000:3.0:1789762967.689858:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0602c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.689867:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0602c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.689874:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0602c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.689879:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.689883:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0602c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.689889:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0602c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.689904:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0602c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.689909:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0602c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.689984:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad02a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015173248:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11287us (47056us total) trans 4294968339 rc 0/0 00000100:00100000:3.0:1789762967.689993:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1458 00000100:00100000:3.0:1789762967.690001:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490982457304 time 490982576825 state 0x2 ref 0 00000100:00100000:2.0:1789762967.692354:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762967.692407:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015173376 00000100:00100000:1.0:1789762967.692416:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015173376 00000100:00100000:1.0:1789762967.692461:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 490982457304 time 490982576825 state 0x0 ref 1 00000100:00100000:1.0:1789762967.692488:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491032576825 time 490982576825 state 0x1 ref 1 00000100:00100000:2.0:1789762967.728796:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491032576825 time 491032696475 state 0x0 ref 1 00000100:00100000:2.0:1789762967.728815:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015173376:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762967.728856:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 74448896, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x23809 00010000:00010000:2.0:1789762967.728863:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b06033 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762967.728875:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:2.0:1789762967.728879:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762967.728883:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b06033 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762967.728895:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b06033 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762967.741059:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968340 is committed 00002000:00010000:3.0:1789762967.741129:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 74448896 -> 75497472 00002000:00010000:3.0:1789762967.741134:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 145417 -> 147465 00010000:00010000:3.0:1789762967.741143:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b06033 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.741163:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b06033 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.741169:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b06033 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.741174:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.741178:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b06033 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.741185:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b06033 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.741198:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b06033 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.741208:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000631598f0/0x416193e012b06033 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.741262:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015173376:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12452us (48907us total) trans 4294968340 rc 0/0 00000100:00100000:3.0:1789762967.741270:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1459 00000100:00100000:3.0:1789762967.741277:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491032576825 time 491032696475 state 0x2 ref 0 00000100:00100000:0.0:1789762967.743499:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762967.743530:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015173504 00000100:00100000:3.0:1789762967.743538:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015173504 00000100:00100000:3.0:1789762967.743572:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491032576825 time 491032696475 state 0x0 ref 1 00000100:00100000:3.0:1789762967.743583:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491082696475 time 491032696475 state 0x1 ref 1 00000100:00100000:1.0:1789762967.778923:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491082696475 time 491082816685 state 0x0 ref 1 00000100:00100000:1.0:1789762967.778956:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015173504:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762967.779007:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 75497472, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x24009 00010000:00010000:1.0:1789762967.779017:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0603a lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.779034:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:1.0:1789762967.779039:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762967.779047:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0603a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.779064:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0603a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762967.791439:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968341 is committed 00002000:00010000:3.0:1789762967.791503:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 75497472 -> 76546048 00002000:00010000:3.0:1789762967.791508:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 147465 -> 149513 00010000:00010000:3.0:1789762967.791520:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0603a lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.791531:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0603a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.791538:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0603a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.791547:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.791551:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0603a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.791558:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0603a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.791574:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0603a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.791579:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000073a02462/0x416193e012b0603a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.791652:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015173504:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12705us (48151us total) trans 4294968341 rc 0/0 00000100:00100000:3.0:1789762967.791661:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1460 00000100:00100000:3.0:1789762967.791670:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491082696475 time 491082816685 state 0x2 ref 0 00000100:00100000:0.0:1789762967.794187:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762967.794224:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015173632 00000100:00100000:3.0:1789762967.794233:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015173632 00000100:00100000:3.0:1789762967.794270:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491082696475 time 491082816685 state 0x0 ref 1 00000100:00100000:3.0:1789762967.794281:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491132816685 time 491082816685 state 0x1 ref 1 00000100:00100000:1.0:1789762967.828997:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491132816685 time 491132893944 state 0x0 ref 1 00000100:00100000:1.0:1789762967.829026:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015173632:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762967.829131:0:17497:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 76546048, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x24809 00010000:00010000:1.0:1789762967.829141:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b06041 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.829159:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:1.0:1789762967.829164:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762967.829170:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b06041 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.829186:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b06041 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762967.840572:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968342 is committed 00002000:00010000:3.0:1789762967.840628:0:17497:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 76546048 -> 77594624 00002000:00010000:3.0:1789762967.840634:0:17497:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 149513 -> 151561 00010000:00010000:3.0:1789762967.840643:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b06041 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.840652:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b06041 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.840658:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b06041 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.840663:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.840666:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b06041 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.840676:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b06041 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.840696:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b06041 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.840706:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cfcbcc1/0x416193e012b06041 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.840771:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad01c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015173632:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11755us (46582us total) trans 4294968342 rc 0/0 00000100:00100000:3.0:1789762967.840782:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1461 00000100:00100000:3.0:1789762967.840795:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491132816685 time 491132893944 state 0x2 ref 0 00000100:00100000:2.0:1789762967.843216:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762967.843251:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015173760 00000100:00100000:3.0:1789762967.843259:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015173760 00000100:00100000:3.0:1789762967.843297:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491132816685 time 491132893944 state 0x0 ref 1 00000100:00100000:3.0:1789762967.843308:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491182893944 time 491132893944 state 0x1 ref 1 00000100:00100000:1.0:1789762967.879072:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491182893944 time 491182966830 state 0x0 ref 1 00000100:00100000:1.0:1789762967.879130:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015173760:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762967.879175:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 77594624, mtime: 0x6aad9d97, atime: 0x0, ctime: 0x6aad9d97, blocks: 0x25009 00010000:00010000:1.0:1789762967.879182:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b06048 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.879195:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:1.0:1789762967.879198:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762967.879203:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b06048 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.879214:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b06048 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762967.890112:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968343 is committed 00002000:00010000:1.0:1789762967.890146:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 77594624 -> 78643200 00002000:00010000:1.0:1789762967.890151:0:12581:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb mtime from disk: 1789762967 -> 1789762968 00002000:00010000:1.0:1789762967.890154:0:12581:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb ctime from disk: 1789762967 -> 1789762968 00002000:00010000:1.0:1789762967.890156:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 151561 -> 153609 00010000:00010000:1.0:1789762967.890165:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b06048 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.890179:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b06048 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.890186:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b06048 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.890190:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762967.890194:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b06048 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.890201:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b06048 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.890215:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b06048 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762967.890221:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b06048 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762967.890281:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015173760:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11191us (47064us total) trans 4294968343 rc 0/0 00000100:00100000:1.0:1789762967.890288:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1462 00000100:00100000:1.0:1789762967.890297:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491182893944 time 491182966830 state 0x2 ref 0 00000100:00100000:0.0:1789762967.892780:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762967.892844:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015173888 00000100:00100000:3.0:1789762967.892854:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015173888 00000100:00100000:3.0:1789762967.892902:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491182893944 time 491182966830 state 0x0 ref 1 00000100:00100000:3.0:1789762967.892919:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491232966830 time 491182966830 state 0x1 ref 1 00000100:00100000:3.0:1789762967.929141:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491232966830 time 491233037109 state 0x0 ref 1 00000100:00100000:3.0:1789762967.929165:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebad00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015173888:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.929211:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 78643200, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x25809 00010000:00010000:3.0:1789762967.929218:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0604f lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.929230:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:3.0:1789762967.929233:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.929238:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0604f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.929250:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0604f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762967.941238:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968344 is committed 00002000:00010000:3.0:1789762967.941309:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 78643200 -> 79691776 00002000:00010000:3.0:1789762967.941314:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 153609 -> 155657 00010000:00010000:3.0:1789762967.941323:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0604f lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.941333:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0604f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.941340:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0604f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.941345:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.941348:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0604f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.941355:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0604f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.941369:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0604f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.941375:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0604f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.941439:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebad00e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015173888:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12282us (48659us total) trans 4294968344 rc 0/0 00000100:00100000:3.0:1789762967.941449:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1463 00000100:00100000:3.0:1789762967.941457:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491232966830 time 491233037109 state 0x2 ref 0 00000100:00100000:2.0:1789762967.943890:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762967.943941:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015174016 00000100:00100000:1.0:1789762967.943949:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015174016 00000100:00100000:1.0:1789762967.943987:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491232966830 time 491233037109 state 0x0 ref 1 00000100:00100000:1.0:1789762967.943998:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491283037109 time 491233037109 state 0x1 ref 1 00000100:00100000:3.0:1789762967.979226:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491283037109 time 491283121608 state 0x0 ref 1 00000100:00100000:3.0:1789762967.979250:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015174016:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762967.979298:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 79691776, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x26009 00010000:00010000:3.0:1789762967.979305:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06056 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.979317:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:3.0:1789762967.979321:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762967.979326:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06056 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.979336:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06056 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762967.990299:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968345 is committed 00002000:00010000:3.0:1789762967.990331:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 79691776 -> 80740352 00002000:00010000:3.0:1789762967.990338:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 155657 -> 157705 00010000:00010000:3.0:1789762967.990352:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06056 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.990367:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06056 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.990384:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06056 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.990393:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762967.990398:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06056 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.990409:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06056 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.990428:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06056 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762967.990437:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06056 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762967.990517:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015174016:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11274us (46625us total) trans 4294968345 rc 0/0 00000100:00100000:3.0:1789762967.990527:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1464 00000100:00100000:3.0:1789762967.990537:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491283037109 time 491283121608 state 0x2 ref 0 00000100:00100000:2.0:1789762967.993164:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762967.993208:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015174144 00000100:00100000:3.0:1789762967.993217:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015174144 00000100:00100000:3.0:1789762967.993262:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491283037109 time 491283121608 state 0x0 ref 1 00000100:00100000:3.0:1789762967.993279:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491333121608 time 491283121608 state 0x1 ref 1 00000100:00100000:3.0:1789762968.029288:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491333121608 time 491333185031 state 0x0 ref 1 00000100:00100000:3.0:1789762968.029309:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015174144:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762968.029374:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 80740352, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x26809 00010000:00010000:3.0:1789762968.029384:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0605d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.029401:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:3.0:1789762968.029404:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762968.029410:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0605d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.029431:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0605d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762968.040559:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968346 is committed 00002000:00010000:3.0:1789762968.040617:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 80740352 -> 81788928 00002000:00010000:3.0:1789762968.040622:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 157705 -> 159753 00010000:00010000:3.0:1789762968.040631:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0605d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.040641:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0605d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.040652:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0605d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.040658:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.040661:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0605d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.040667:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0605d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.040681:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0605d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.040686:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0605d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.040742:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015174144:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11438us (47576us total) trans 4294968346 rc 0/0 00000100:00100000:3.0:1789762968.040749:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1465 00000100:00100000:3.0:1789762968.040758:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491333121608 time 491333185031 state 0x2 ref 0 00000100:00100000:0.0:1789762968.043028:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762968.043108:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015174272 00000100:00100000:2.0:1789762968.043132:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015174272 00000100:00100000:2.0:1789762968.043170:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491333121608 time 491333185031 state 0x0 ref 1 00000100:00100000:2.0:1789762968.043180:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491383185031 time 491333185031 state 0x1 ref 1 00000100:00100000:3.0:1789762968.079400:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491383185031 time 491383296040 state 0x0 ref 1 00000100:00100000:3.0:1789762968.079432:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82087b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015174272:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762968.079480:0:17498:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 81788928, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x27009 00010000:00010000:3.0:1789762968.079487:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06064 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.079500:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:3.0:1789762968.079504:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762968.079509:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06064 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.079519:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06064 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762968.091070:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968347 is committed 00002000:00010000:3.0:1789762968.091105:0:17498:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 81788928 -> 82837504 00002000:00010000:3.0:1789762968.091124:0:17498:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 159753 -> 161801 00010000:00010000:3.0:1789762968.091138:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06064 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.091152:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06064 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.091163:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06064 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.091171:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.091174:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06064 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.091186:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06064 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.091200:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06064 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.091206:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06064 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.091286:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82087b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015174272:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11866us (48253us total) trans 4294968347 rc 0/0 00000100:00100000:3.0:1789762968.091298:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1466 00000100:00100000:3.0:1789762968.091309:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491383185031 time 491383296040 state 0x2 ref 0 00000100:00100000:0.0:1789762968.093812:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762968.093866:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015174400 00000100:00100000:2.0:1789762968.093876:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015174400 00000100:00100000:2.0:1789762968.093926:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491383185031 time 491383296040 state 0x0 ref 1 00000100:00100000:2.0:1789762968.093941:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491433296040 time 491383296040 state 0x1 ref 1 00000100:00100000:1.0:1789762968.129455:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491433296040 time 491433352866 state 0x0 ref 1 00000100:00100000:1.0:1789762968.129475:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015174400:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762968.129516:0:14913:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 82837504, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x27809 00010000:00010000:1.0:1789762968.129523:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0606b lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.129536:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:1.0:1789762968.129539:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762968.129544:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0606b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.129554:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0606b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762968.140448:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968348 is committed 00002000:00010000:3.0:1789762968.140472:0:14913:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 82837504 -> 83886080 00002000:00010000:3.0:1789762968.140478:0:14913:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 161801 -> 163849 00010000:00010000:3.0:1789762968.140487:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0606b lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.140500:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0606b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.140510:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0606b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.140525:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.140530:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0606b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.140541:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0606b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.140557:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0606b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.140563:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004e67eb4a/0x416193e012b0606b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.140631:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb27800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015174400:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11160us (46820us total) trans 4294968348 rc 0/0 00000100:00100000:3.0:1789762968.140643:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1467 00000100:00100000:3.0:1789762968.140654:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491433296040 time 491433352866 state 0x2 ref 0 00000100:00100000:0.0:1789762968.142953:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762968.142988:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015174528 00000100:00100000:3.0:1789762968.142996:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015174528 00000100:00100000:3.0:1789762968.143032:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491433296040 time 491433352866 state 0x0 ref 1 00000100:00100000:3.0:1789762968.143043:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491483352866 time 491433352866 state 0x1 ref 1 00000100:00100000:3.0:1789762968.179509:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491483352866 time 491483402652 state 0x0 ref 1 00000100:00100000:3.0:1789762968.179538:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebac27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015174528:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762968.179584:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 83886080, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x28009 00010000:00010000:3.0:1789762968.179600:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06072 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.179613:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:3.0:1789762968.179616:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762968.179622:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06072 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.179632:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06072 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762968.191323:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968349 is committed 00002000:00010000:3.0:1789762968.191353:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 83886080 -> 84934656 00002000:00010000:3.0:1789762968.191359:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 163849 -> 165897 00010000:00010000:3.0:1789762968.191369:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06072 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.191380:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06072 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.191387:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06072 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.191392:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.191396:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06072 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.191403:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06072 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.191418:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06072 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.191423:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06072 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.191506:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebac27100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015174528:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11976us (48550us total) trans 4294968349 rc 0/0 00000100:00100000:3.0:1789762968.191520:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1468 00000100:00100000:3.0:1789762968.191533:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491483352866 time 491483402652 state 0x2 ref 0 00000100:00100000:0.0:1789762968.194225:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762968.194277:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015174656 00000100:00100000:1.0:1789762968.194286:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015174656 00000100:00100000:1.0:1789762968.194327:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491483352866 time 491483402652 state 0x0 ref 1 00000100:00100000:1.0:1789762968.194341:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491533402652 time 491483402652 state 0x1 ref 1 00000100:00100000:2.0:1789762968.229588:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491533402652 time 491533484097 state 0x0 ref 1 00000100:00100000:2.0:1789762968.229614:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015174656:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762968.229663:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 84934656, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x28809 00010000:00010000:2.0:1789762968.229671:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bf1be8d9/0x416193e012b06079 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762968.229684:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:2.0:1789762968.229687:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762968.229692:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bf1be8d9/0x416193e012b06079 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762968.229720:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bf1be8d9/0x416193e012b06079 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762968.240891:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968350 is committed 00002000:00010000:3.0:1789762968.240908:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 84934656 -> 85983232 00002000:00010000:3.0:1789762968.240914:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 165897 -> 167945 00010000:00010000:3.0:1789762968.240925:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bf1be8d9/0x416193e012b06079 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.240935:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bf1be8d9/0x416193e012b06079 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.240941:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bf1be8d9/0x416193e012b06079 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.240951:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.240955:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf1be8d9/0x416193e012b06079 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.240961:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bf1be8d9/0x416193e012b06079 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.240975:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bf1be8d9/0x416193e012b06079 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.240980:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bf1be8d9/0x416193e012b06079 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.241041:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015174656:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11436us (46816us total) trans 4294968350 rc 0/0 00000100:00100000:3.0:1789762968.241050:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1469 00000100:00100000:3.0:1789762968.241060:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491533402652 time 491533484097 state 0x2 ref 0 00000100:00100000:0.0:1789762968.243523:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762968.243592:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015174784 00000100:00100000:1.0:1789762968.243602:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015174784 00000100:00100000:1.0:1789762968.243647:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491533402652 time 491533484097 state 0x0 ref 1 00000100:00100000:1.0:1789762968.243664:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491583484097 time 491533484097 state 0x1 ref 1 00000100:00100000:2.0:1789762968.279709:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491583484097 time 491583605490 state 0x0 ref 1 00000100:00100000:2.0:1789762968.279732:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d82f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015174784:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762968.279803:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 85983232, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x29009 00010000:00010000:2.0:1789762968.279813:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b06080 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762968.279829:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:2.0:1789762968.279834:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762968.279841:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b06080 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762968.279854:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b06080 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762968.291481:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968351 is committed 00002000:00010000:3.0:1789762968.291523:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 85983232 -> 87031808 00002000:00010000:3.0:1789762968.291530:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 167945 -> 169993 00010000:00010000:3.0:1789762968.291543:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b06080 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.291556:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b06080 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.291573:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b06080 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.291583:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc93c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.291588:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b06080 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.291599:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b06080 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.291619:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b06080 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.291628:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008e23d646/0x416193e012b06080 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.291702:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d82f93480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015174784:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11975us (48177us total) trans 4294968351 rc 0/0 00000100:00100000:3.0:1789762968.291714:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1470 00000100:00100000:3.0:1789762968.291727:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491583484097 time 491583605490 state 0x2 ref 0 00000100:00100000:0.0:1789762968.294156:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762968.294195:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015174912 00000100:00100000:3.0:1789762968.294205:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015174912 00000100:00100000:3.0:1789762968.294253:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491583484097 time 491583605490 state 0x0 ref 1 00000100:00100000:3.0:1789762968.294273:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491633605490 time 491583605490 state 0x1 ref 1 00000100:00100000:3.0:1789762968.329785:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491633605490 time 491633676397 state 0x0 ref 1 00000100:00100000:3.0:1789762968.329815:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015174912:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762968.329868:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 87031808, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x29809 00010000:00010000:3.0:1789762968.329879:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06087 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.329896:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:3.0:1789762968.329902:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762968.329908:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06087 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.329924:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06087 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762968.342830:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968352 is committed 00002000:00010000:3.0:1789762968.342843:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 87031808 -> 88080384 00002000:00010000:3.0:1789762968.342848:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 169993 -> 172041 00010000:00010000:3.0:1789762968.342861:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06087 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.342872:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06087 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.342883:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06087 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.342889:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.342892:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06087 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.342898:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06087 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.342913:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06087 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.342918:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b06087 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.342986:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbc9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015174912:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13181us (48828us total) trans 4294968352 rc 0/0 00000100:00100000:3.0:1789762968.342994:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1471 00000100:00100000:3.0:1789762968.343003:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491633605490 time 491633676397 state 0x2 ref 0 00000100:00100000:0.0:1789762968.345500:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762968.345554:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015175040 00000100:00100000:1.0:1789762968.345564:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015175040 00000100:00100000:1.0:1789762968.345612:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491633605490 time 491633676397 state 0x0 ref 1 00000100:00100000:1.0:1789762968.345625:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491683676397 time 491633676397 state 0x1 ref 1 00000100:00100000:1.0:1789762968.379856:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491683676397 time 491683752137 state 0x0 ref 1 00000100:00100000:1.0:1789762968.379898:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015175040:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762968.379947:0:17497:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 88080384, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x2a009 00010000:00010000:1.0:1789762968.379955:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0608e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.379969:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:1.0:1789762968.379972:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762968.379977:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0608e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.379989:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0608e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762968.392178:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968353 is committed 00002000:00010000:3.0:1789762968.392247:0:17497:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 88080384 -> 89128960 00002000:00010000:3.0:1789762968.392252:0:17497:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 172041 -> 174089 00010000:00010000:3.0:1789762968.392264:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0608e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.392280:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0608e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.392287:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0608e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.392292:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.392296:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0608e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.392302:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0608e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.392318:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0608e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.392324:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008a54841d/0x416193e012b0608e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.392408:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbc9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015175040:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12516us (46905us total) trans 4294968353 rc 0/0 00000100:00100000:3.0:1789762968.392418:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1472 00000100:00100000:3.0:1789762968.392432:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491683676397 time 491683752137 state 0x2 ref 0 00000100:00100000:0.0:1789762968.395449:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762968.395484:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015175168 00000100:00100000:3.0:1789762968.395494:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015175168 00000100:00100000:3.0:1789762968.395546:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491683676397 time 491683752137 state 0x0 ref 1 00000100:00100000:3.0:1789762968.395558:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491733752137 time 491683752137 state 0x1 ref 1 00000100:00100000:1.0:1789762968.429950:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491733752137 time 491733845564 state 0x0 ref 1 00000100:00100000:1.0:1789762968.429974:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015175168:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762968.430027:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 89128960, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x2a809 00010000:00010000:1.0:1789762968.430037:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06095 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.430051:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:1.0:1789762968.430055:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762968.430060:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06095 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.430072:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06095 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762968.443013:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968354 is committed 00002000:00010000:3.0:1789762968.443083:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 89128960 -> 90177536 00002000:00010000:3.0:1789762968.443107:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 174089 -> 176137 00010000:00010000:3.0:1789762968.443133:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06095 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.443147:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06095 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.443156:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06095 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.443164:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.443169:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06095 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.443178:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06095 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.443197:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06095 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.443206:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06095 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.443290:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbc8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015175168:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13320us (47839us total) trans 4294968354 rc 0/0 00000100:00100000:3.0:1789762968.443302:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1473 00000100:00100000:3.0:1789762968.443315:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491733752137 time 491733845564 state 0x2 ref 0 00000100:00100000:2.0:1789762968.446366:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762968.446400:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015175296 00000100:00100000:3.0:1789762968.446408:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015175296 00000100:00100000:3.0:1789762968.446446:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491733752137 time 491733845564 state 0x0 ref 1 00000100:00100000:3.0:1789762968.446457:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491783845564 time 491733845564 state 0x1 ref 1 00000100:00100000:3.0:1789762968.480012:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491783845564 time 491783907613 state 0x0 ref 1 00000100:00100000:3.0:1789762968.480035:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015175296:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762968.480081:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 90177536, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x2b009 00010000:00010000:3.0:1789762968.480121:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0609c lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.480137:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:3.0:1789762968.480141:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762968.480146:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0609c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.480164:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0609c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762968.493039:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968355 is committed 00002000:00010000:3.0:1789762968.493052:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 90177536 -> 91226112 00002000:00010000:3.0:1789762968.493058:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 176137 -> 178185 00010000:00010000:3.0:1789762968.493080:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0609c lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.493124:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0609c lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.493133:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0609c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.493139:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.493143:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0609c lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.493160:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0609c lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.493176:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0609c lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.493182:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b0609c lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.493271:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015175296:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13238us (46903us total) trans 4294968355 rc 0/0 00000100:00100000:3.0:1789762968.493282:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1474 00000100:00100000:3.0:1789762968.493296:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491783845564 time 491783907613 state 0x2 ref 0 00000100:00100000:2.0:1789762968.496427:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762968.496482:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015175424 00000100:00100000:1.0:1789762968.496490:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015175424 00000100:00100000:1.0:1789762968.496543:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491783845564 time 491783907613 state 0x0 ref 1 00000100:00100000:1.0:1789762968.496559:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491833907613 time 491783907613 state 0x1 ref 1 00010000:00010000:0.0:1789762968.503201:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762968.530153:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491833907613 time 491834044863 state 0x0 ref 1 00000100:00100000:1.0:1789762968.530184:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015175424:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762968.530237:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 91226112, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x2b809 00010000:00010000:1.0:1789762968.530248:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b060a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.530264:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:1.0:1789762968.530269:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762968.530276:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b060a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.530292:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b060a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762968.543026:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968356 is committed 00002000:00010000:3.0:1789762968.543029:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 91226112 -> 92274688 00002000:00010000:3.0:1789762968.543034:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 178185 -> 180233 00010000:00010000:3.0:1789762968.543047:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b060a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.543057:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b060a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.543065:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b060a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.543074:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.543079:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b060a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.543127:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b060a3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.543144:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b060a3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.543150:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbfb607a/0x416193e012b060a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.543223:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015175424:12345-192.168.203.48@tcp:4:dd.500 Request processed in 13051us (46797us total) trans 4294968356 rc 0/0 00000100:00100000:3.0:1789762968.543232:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1475 00000100:00100000:3.0:1789762968.543241:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491833907613 time 491834044863 state 0x2 ref 0 00000100:00100000:2.0:1789762968.546322:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762968.546378:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015175552 00000100:00100000:3.0:1789762968.546385:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015175552 00000100:00100000:3.0:1789762968.546424:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491833907613 time 491834044863 state 0x0 ref 1 00000100:00100000:3.0:1789762968.546435:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491884044863 time 491834044863 state 0x1 ref 1 00000100:00100000:3.0:1789762968.580220:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491884044863 time 491884115040 state 0x0 ref 1 00000100:00100000:3.0:1789762968.580244:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015175552:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762968.580293:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 92274688, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x2c009 00010000:00010000:3.0:1789762968.580300:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060aa lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.580314:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:3.0:1789762968.580318:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762968.580322:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.580333:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789762968.592002:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968357 is committed 00002000:00010000:3.0:1789762968.592060:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 92274688 -> 93323264 00002000:00010000:3.0:1789762968.592065:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 180233 -> 182281 00010000:00010000:3.0:1789762968.592075:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060aa lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.592102:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060aa lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.592121:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.592128:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.592133:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060aa lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.592139:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060aa lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.592155:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060aa lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.592160:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060aa lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.592226:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015175552:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11989us (45905us total) trans 4294968357 rc 0/0 00000100:00100000:3.0:1789762968.592235:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1476 00000100:00100000:3.0:1789762968.592244:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491884044863 time 491884115040 state 0x2 ref 0 00000100:00100000:0.0:1789762968.594749:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762968.594794:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015175680 00000100:00100000:2.0:1789762968.594803:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015175680 00000100:00100000:2.0:1789762968.594843:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491884044863 time 491884115040 state 0x0 ref 1 00000100:00100000:2.0:1789762968.594855:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491934115040 time 491884115040 state 0x1 ref 1 00000100:00100000:3.0:1789762968.630293:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491934115040 time 491934194241 state 0x0 ref 1 00000100:00100000:3.0:1789762968.630310:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015175680:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762968.630347:0:17498:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 93323264, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x2c809 00010000:00010000:3.0:1789762968.630353:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b1 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.630366:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:3.0:1789762968.630369:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762968.630374:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.630383:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762968.641452:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968358 is committed 00002000:00010000:3.0:1789762968.641515:0:17498:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 93323264 -> 94371840 00002000:00010000:3.0:1789762968.641520:0:17498:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 182281 -> 184329 00010000:00010000:3.0:1789762968.641531:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b1 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.641541:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b1 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.641549:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.641554:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.641557:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b1 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.641563:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b1 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.641577:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b1 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.641582:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b1 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.641651:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015175680:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11343us (46900us total) trans 4294968358 rc 0/0 00000100:00100000:3.0:1789762968.641659:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1477 00000100:00100000:3.0:1789762968.641668:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491934115040 time 491934194241 state 0x2 ref 0 00000100:00100000:0.0:1789762968.644279:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762968.644321:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015175808 00000100:00100000:2.0:1789762968.644331:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015175808 00000100:00100000:2.0:1789762968.644377:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491934115040 time 491934194241 state 0x0 ref 1 00000100:00100000:2.0:1789762968.644391:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491984194241 time 491934194241 state 0x1 ref 1 00000100:00100000:3.0:1789762968.680391:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491984194241 time 491984288308 state 0x0 ref 1 00000100:00100000:3.0:1789762968.680415:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015175808:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762968.680475:0:14912:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 94371840, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x2d009 00010000:00010000:3.0:1789762968.680482:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.680495:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=491s v=(0 0) 00010000:00010000:3.0:1789762968.680498:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762968.680503:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.680514:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762968.691804:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968359 is committed 00002000:00010000:3.0:1789762968.691820:0:14912:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 94371840 -> 95420416 00002000:00010000:3.0:1789762968.691825:0:14912:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 184329 -> 186379 00010000:00010000:3.0:1789762968.691836:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.691846:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.691852:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.691858:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.691861:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.691874:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.691893:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.691903:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060b8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.691984:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015175808:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11577us (47704us total) trans 4294968359 rc 0/0 00000100:00100000:3.0:1789762968.691994:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1478 00000100:00100000:3.0:1789762968.692003:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491984194241 time 491984288308 state 0x2 ref 0 00000100:00100000:2.0:1789762968.694385:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762968.694473:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015175936 00000100:00100000:1.0:1789762968.694482:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015175936 00000100:00100000:1.0:1789762968.694521:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 491984194241 time 491984288308 state 0x0 ref 1 00000100:00100000:1.0:1789762968.694532:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492034288308 time 491984288308 state 0x1 ref 1 00000001:00080000:3.0:1789762968.708000:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967448 is committed 00000001:00080000:3.0:1789762968.708040:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967451 is committed 00000001:00080000:3.0:1789762968.708043:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967450 is committed 00000100:00100000:3.0:1789762968.730457:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492034288308 time 492034354609 state 0x0 ref 1 00000100:00100000:3.0:1789762968.730480:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc587b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015175936:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762968.730536:0:14914:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 95420416, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x2d80b 00010000:00010000:3.0:1789762968.730554:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060bf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.730577:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=492s v=(0 0) 00010000:00010000:3.0:1789762968.730582:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762968.730589:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.730604:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762968.742238:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968360 is committed 00002000:00010000:3.0:1789762968.742254:0:14914:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 95420416 -> 96468992 00002000:00010000:3.0:1789762968.742259:0:14914:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 186379 -> 188427 00010000:00010000:3.0:1789762968.742271:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060bf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.742281:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060bf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.742287:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.742292:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.742296:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060bf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.742303:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060bf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.742321:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060bf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.742327:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060bf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.742388:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc587b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015175936:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11915us (48003us total) trans 4294968360 rc 0/0 00000100:00100000:3.0:1789762968.742396:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1479 00000100:00100000:3.0:1789762968.742406:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492034288308 time 492034354609 state 0x2 ref 0 00000100:00100000:0.0:1789762968.744986:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762968.745067:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015176064 00000100:00100000:1.0:1789762968.745076:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015176064 00000100:00100000:1.0:1789762968.745153:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492034288308 time 492034354609 state 0x0 ref 1 00000100:00100000:1.0:1789762968.745165:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492084354609 time 492034354609 state 0x1 ref 1 00000100:00100000:2.0:1789762968.780530:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492084354609 time 492084429120 state 0x0 ref 1 00000100:00100000:2.0:1789762968.780550:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015176064:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762968.780595:0:12582:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 96468992, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x2e00b 00010000:00010000:2.0:1789762968.780602:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b060c6 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762968.780615:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=492s v=(0 0) 00010000:00010000:2.0:1789762968.780619:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762968.780623:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b060c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762968.780650:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b060c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762968.792071:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968361 is committed 00002000:00010000:3.0:1789762968.792147:0:12582:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 96468992 -> 97517568 00002000:00010000:3.0:1789762968.792154:0:12582:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 188427 -> 190475 00010000:00010000:3.0:1789762968.792167:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b060c6 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.792179:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b060c6 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.792188:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b060c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.792195:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.792200:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b060c6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.792215:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b060c6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.792234:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b060c6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.792242:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007d465e24/0x416193e012b060c6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.792325:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015176064:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11777us (47337us total) trans 4294968361 rc 0/0 00000100:00100000:3.0:1789762968.792338:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1480 00000100:00100000:3.0:1789762968.792351:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492084354609 time 492084429120 state 0x2 ref 0 00000100:00100000:0.0:1789762968.794732:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762968.794765:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015176192 00000100:00100000:3.0:1789762968.794774:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015176192 00000100:00100000:3.0:1789762968.794818:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492084354609 time 492084429120 state 0x0 ref 1 00000100:00100000:3.0:1789762968.794833:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492134429120 time 492084429120 state 0x1 ref 1 00000100:00100000:2.0:1789762968.830652:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492134429120 time 492134541624 state 0x0 ref 1 00000100:00100000:2.0:1789762968.830679:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015176192:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:2.0:1789762968.830739:0:6654:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 97517568, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x2e80b 00010000:00010000:2.0:1789762968.830747:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b060cd lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762968.830760:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=492s v=(0 0) 00010000:00010000:2.0:1789762968.830764:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762968.830770:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b060cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762968.830785:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b060cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762968.842376:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968362 is committed 00002000:00010000:3.0:1789762968.842412:0:6654:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 97517568 -> 98566144 00002000:00010000:3.0:1789762968.842417:0:6654:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 190475 -> 192523 00010000:00010000:3.0:1789762968.842429:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b060cd lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.842445:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b060cd lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.842451:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b060cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.842456:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfc92600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.842460:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b060cd lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.842466:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b060cd lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.842480:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b060cd lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.842485:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000896187e3/0x416193e012b060cd lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.842550:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015176192:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11880us (47818us total) trans 4294968362 rc 0/0 00000100:00100000:3.0:1789762968.842558:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1481 00000100:00100000:3.0:1789762968.842567:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492134429120 time 492134541624 state 0x2 ref 0 00000100:00100000:3.0:1789762968.845068:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762968.845133:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015176320 00000100:00100000:1.0:1789762968.845142:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015176320 00000100:00100000:1.0:1789762968.845193:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492134429120 time 492134541624 state 0x0 ref 1 00000100:00100000:1.0:1789762968.845204:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492184541624 time 492134541624 state 0x1 ref 1 00000100:00100000:1.0:1789762968.880718:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492184541624 time 492184615302 state 0x0 ref 1 00000100:00100000:1.0:1789762968.880742:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015176320:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762968.880802:0:14913:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 98566144, mtime: 0x6aad9d98, atime: 0x0, ctime: 0x6aad9d98, blocks: 0x2f00b 00010000:00010000:1.0:1789762968.880810:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060d4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.880823:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=492s v=(0 0) 00010000:00010000:1.0:1789762968.880826:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762968.880831:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.880842:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762968.892685:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968363 is committed 00002000:00010000:1.0:1789762968.892690:0:14913:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 98566144 -> 99614720 00002000:00010000:1.0:1789762968.892695:0:14913:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb mtime from disk: 1789762968 -> 1789762969 00002000:00010000:1.0:1789762968.892698:0:14913:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb ctime from disk: 1789762968 -> 1789762969 00002000:00010000:1.0:1789762968.892700:0:14913:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 192523 -> 194571 00010000:00010000:1.0:1789762968.892709:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060d4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.892719:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060d4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.892726:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.892731:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762968.892734:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060d4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.892740:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060d4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.892754:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060d4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.892760:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060d4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762968.892822:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015176320:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12087us (47752us total) trans 4294968363 rc 0/0 00000100:00100000:1.0:1789762968.892831:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1482 00000100:00100000:1.0:1789762968.892840:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492184541624 time 492184615302 state 0x2 ref 0 00000100:00100000:0.0:1789762968.895183:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762968.895224:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015176448 00000100:00100000:3.0:1789762968.895233:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015176448 00000100:00100000:3.0:1789762968.895271:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492184541624 time 492184615302 state 0x0 ref 1 00000100:00100000:3.0:1789762968.895282:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492234615302 time 492184615302 state 0x1 ref 1 00000100:00100000:1.0:1789762968.930818:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492234615302 time 492234718164 state 0x0 ref 1 00000100:00100000:1.0:1789762968.930842:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015176448:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:1.0:1789762968.930882:0:17497:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 99614720, mtime: 0x6aad9d99, atime: 0x0, ctime: 0x6aad9d99, blocks: 0x2f80b 00010000:00010000:1.0:1789762968.930888:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060db lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.930900:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=492s v=(0 0) 00010000:00010000:1.0:1789762968.930904:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762968.930908:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.930919:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789762968.942554:0:17497:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 99614720 -> 100663296 00000001:00080000:2.0:1789762968.942559:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968364 is committed 00002000:00010000:1.0:1789762968.942560:0:17497:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 194571 -> 196619 00010000:00010000:1.0:1789762968.942573:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060db lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.942583:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060db lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.942589:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.942595:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762968.942599:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060db lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.942605:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060db lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.942631:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060db lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762968.942639:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060db lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762968.942701:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015176448:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11863us (47517us total) trans 4294968364 rc 0/0 00000100:00100000:1.0:1789762968.942710:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1483 00000100:00100000:1.0:1789762968.942719:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492234615302 time 492234718164 state 0x2 ref 0 00000100:00100000:0.0:1789762968.945000:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762968.945044:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015176576 00000100:00100000:3.0:1789762968.945051:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015176576 00000100:00100000:3.0:1789762968.945102:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492234615302 time 492234718164 state 0x0 ref 1 00000100:00100000:3.0:1789762968.945127:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492284718164 time 492234718164 state 0x1 ref 1 00000100:00100000:3.0:1789762968.980919:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492284718164 time 492284816010 state 0x0 ref 1 00000100:00100000:3.0:1789762968.980941:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015176576:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762968.980985:0:12581:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 100663296, mtime: 0x6aad9d99, atime: 0x0, ctime: 0x6aad9d99, blocks: 0x3000b 00010000:00010000:3.0:1789762968.980992:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.981005:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=492s v=(0 0) 00010000:00010000:3.0:1789762968.981008:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762968.981012:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.981027:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762968.992168:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968365 is committed 00002000:00010000:3.0:1789762968.992228:0:12581:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 100663296 -> 101711872 00002000:00010000:3.0:1789762968.992234:0:12581:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 196619 -> 198667 00010000:00010000:3.0:1789762968.992243:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.992253:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.992260:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.992265:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762968.992269:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.992275:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060e2 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.992290:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060e2 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762968.992295:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762968.992362:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbc9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015176576:12345-192.168.203.48@tcp:4:dd.500 Request processed in 11424us (47358us total) trans 4294968365 rc 0/0 00000100:00100000:3.0:1789762968.992374:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1484 00000100:00100000:3.0:1789762968.992382:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492284718164 time 492284816010 state 0x2 ref 0 00000100:00100000:2.0:1789762968.994564:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762968.994643:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015176704 00000100:00100000:0.0:1789762968.994652:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015176704 00000100:00100000:0.0:1789762968.994688:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492284718164 time 492284816010 state 0x0 ref 1 00000100:00100000:0.0:1789762968.994699:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492334816010 time 492284816010 state 0x1 ref 1 00000100:00100000:1.0:1789762969.030991:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492334816010 time 492334891871 state 0x0 ref 1 00000100:00100000:1.0:1789762969.031021:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc586d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015176704:12345-192.168.203.48@tcp:4:dd.500 00000100:00080000:2.0:1789762969.031053:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789762969.031066:0:6652:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 101711872, mtime: 0x6aad9d99, atime: 0x0, ctime: 0x6aad9d99, blocks: 0x3080b 00010000:00010000:1.0:1789762969.031075:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.031124:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=492s v=(0 0) 00010000:00010000:1.0:1789762969.031128:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762969.031133:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.031144:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762969.041383:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968366 is committed 00002000:00010000:1.0:1789762969.041388:0:6652:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 101711872 -> 102760448 00002000:00010000:1.0:1789762969.041395:0:6652:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 198667 -> 200715 00010000:00010000:1.0:1789762969.041407:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.041420:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.041432:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.041441:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762969.041447:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.041458:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060e9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.041474:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060e9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.041483:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b060e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762969.041542:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc586d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015176704:12345-192.168.203.48@tcp:4:dd.500 Request processed in 10537us (46978us total) trans 4294968366 rc 0/0 00000100:00100000:1.0:1789762969.041553:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1485 00000100:00100000:1.0:1789762969.041563:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492334816010 time 492334891871 state 0x2 ref 0 00000100:00100000:0.0:1789762969.043601:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762969.043641:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015176832 00000100:00100000:3.0:1789762969.043648:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015176832 00000100:00100000:3.0:1789762969.043681:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492334816010 time 492334891871 state 0x0 ref 1 00000100:00100000:3.0:1789762969.043692:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492384891871 time 492334891871 state 0x1 ref 1 00000100:00100000:3.0:1789762969.081032:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492384891871 time 492384931665 state 0x0 ref 1 00000100:00100000:3.0:1789762969.081052:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015176832:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762969.081117:0:6653:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 102760448, mtime: 0x6aad9d99, atime: 0x0, ctime: 0x6aad9d99, blocks: 0x3100b 00010000:00010000:3.0:1789762969.081125:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.081137:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=492s v=(0 0) 00010000:00010000:3.0:1789762969.081141:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762969.081145:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.081156:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789762969.093205:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968367 is committed 00002000:00010000:3.0:1789762969.093213:0:6653:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 102760448 -> 103809024 00002000:00010000:3.0:1789762969.093218:0:6653:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 200715 -> 202763 00010000:00010000:3.0:1789762969.093228:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.093238:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.093244:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.093250:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762969.093253:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.093260:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.093274:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.093279:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762969.093345:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015176832:12345-192.168.203.48@tcp:4:dd.500 Request processed in 12296us (49740us total) trans 4294968367 rc 0/0 00000100:00100000:3.0:1789762969.093358:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1486 00000100:00100000:3.0:1789762969.093369:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492384891871 time 492384931665 state 0x2 ref 0 00000100:00100000:0.0:1789762969.095511:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762969.095560:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015176960 00000100:00100000:2.0:1789762969.095575:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015176960 00000100:00100000:2.0:1789762969.095612:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492384891871 time 492384931665 state 0x0 ref 1 00000100:00100000:2.0:1789762969.095623:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492434931665 time 492384931665 state 0x1 ref 1 00000100:00100000:3.0:1789762969.131233:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492434931665 time 492435114756 state 0x0 ref 1 00000100:00100000:3.0:1789762969.131277:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015176960:12345-192.168.203.48@tcp:4:dd.500 00002000:00010000:3.0:1789762969.131357:0:11723:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 103809024, mtime: 0x6aad9d99, atime: 0x0, ctime: 0x6aad9d99, blocks: 0x3180b 00010000:00010000:3.0:1789762969.131372:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.131398:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=492s v=(0 0) 00010000:00010000:3.0:1789762969.131406:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762969.131419:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.131443:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789762969.146651:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968368 is committed 00002000:00010000:3.0:1789762969.146667:0:11723:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 103809024 -> 104857600 00002000:00010000:3.0:1789762969.146672:0:11723:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 202763 -> 204811 00010000:00010000:3.0:1789762969.146683:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.146692:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.146700:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.146705:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880abe00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762969.146709:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.146715:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.146729:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.146734:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001d729e59/0x416193e012b060f7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762969.146793:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbc8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015176960:12345-192.168.203.48@tcp:4:dd.500 Request processed in 15533us (51281us total) trans 4294968368 rc 0/0 00000100:00100000:3.0:1789762969.146801:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1487 00000100:00100000:3.0:1789762969.146809:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981ebf673200 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_write}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978500 gen 0 rate 20 depth 3 token 0 deadline 492434931665 time 492435114756 state 0x2 ref 0 00000100:00100000:0.0:1789762969.148716:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762969.148764:0:5807:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015177216 00000100:00100000:3.0:1789762969.148772:0:5807:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015177216 00000100:00100000:3.0:1789762969.148823:0:5807:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 129 00000100:00080000:2.0:1789762969.148834:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762969.148835:0:5807:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+8:15873:x1876702015177216:12345-192.168.203.48@tcp:35:dd.500 00000001:00080000:3.0:1789762969.149034:0:5807:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967644 00000100:00100000:3.0:1789762969.149160:0:5807:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15873:x1876702015177216:12345-192.168.203.48@tcp:35:dd.500 Request processed in 310us (426us total) trans 4294967645 rc 0/0 00000100:00100000:3.0:1789762969.149175:0:5807:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 129 00000020:00080000:3.0:1789762969.149183:0:5807:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789762969.173944:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762969.174026:0:5790:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015177344 00000100:00100000:0.0:1789762969.174033:0:5790:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015177344 00000100:00100000:0.0:1789762969.174128:0:5790:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 40 00000100:00100000:0.0:1789762969.174140:0:5790:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc584380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:5593:x1876702015177344:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789762969.174152:0:5790:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:0.0:1789762969.174166:0:5790:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb blocks from disk: 1 -> 16391 00010000:00010000:0.0:1789762969.174185:0:5790:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000020801b7b/0x416193e012b05cd6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd918 expref: 6 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.174203:0:5790:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789762969.174239:0:5790:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc584380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:5593:x1876702015177344:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 103us (302us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762969.174245:0:5790:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 40 00000100:00100000:0.0:1789762969.194239:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762969.194282:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015177600 00000100:00100000:3.0:1789762969.194289:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015177600 00000100:00100000:3.0:1789762969.194333:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 400 00000100:00100000:3.0:1789762969.194341:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15889:x1876702015177600:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762969.194358:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762969.194376:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b060fe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd96c expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.194475:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b06105 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x43:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.194489:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b06105 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x43:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.194499:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b06105 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x43:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762969.194508:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x43:0x0]: returning size 104857600 00010000:00010000:3.0:1789762969.194611:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880abe00/0x416193e012b060fe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x41:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd96c expref: 13 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.194628:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b06105 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x43:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd96c expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.194643:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x43:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:3.0:1789762969.194645:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:3.0:1789762969.194647:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880ab800, rc 0) 00000100:00100000:3.0:1789762969.194681:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbc8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15889:x1876702015177600:12345-192.168.203.48@tcp:101:dd.500 Request processed in 341us (445us total) trans 4294967646 rc 0/0 00000100:00100000:3.0:1789762969.194687:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 400 00000100:00100000:2.0:1789762969.195787:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762969.195821:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015177728 00000100:00100000:3.0:1789762969.195826:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015177728 00000100:00100000:3.0:1789762969.195856:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 229 00000100:00100000:3.0:1789762969.195862:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebc585880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2364:x1876702015177728:12345-192.168.203.48@tcp:101:dd.500 00010000:00010000:3.0:1789762969.195869:0:6647:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789762969.195879:0:6647:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b06113 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd973 expref: -99 pid: 6647 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789762969.195894:0:6647:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 104857600, mtime: 0x6aad9d99, atime: 0x0, ctime: 0x6aad9d99, blocks: 0x3200b 00010000:00010000:3.0:1789762969.195906:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=492s v=(0 0) 00010000:00010000:3.0:1789762969.195911:0:6647:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762969.195917:0:6647:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b06113 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd973 expref: 6 pid: 6647 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.195929:0:6647:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b06113 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd973 expref: 6 pid: 6647 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.195936:0:6647:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e880aae00, rc 0) 00000100:00100000:3.0:1789762969.195963:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebc585880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015177728:12345-192.168.203.48@tcp:101:dd.500 Request processed in 101us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762969.195968:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 229 00000100:00100000:2.0:1789762969.198238:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762969.198272:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015177856 00000100:00100000:3.0:1789762969.198277:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015177856 00000100:00100000:3.0:1789762969.198307:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 472385101385 time 472385220662 state 0x0 ref 1 00000100:00100000:3.0:1789762969.198314:0:12582:0:(nrs_tbf.c:2662:nrs_tbf_res_get()) ost_io.0 NRS: TBF rule update, rule flags 0x1, head sequence 11: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d81d4ab00 gen 0 rate 5 depth 3 token 0 deadline 472385101385 time 472385220662 state 0x0 ref 1 00000100:00100000:3.0:1789762969.198331:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 3 deadline 492602238313 time 492502238313 state 0x1 ref 1 00000100:00100000:3.0:1789762969.198353:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 2 deadline 492602238313 time 492502261606 state 0x0 ref 1 00000100:00100000:3.0:1789762969.198362:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015177856:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762969.198381:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0611a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.198389:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762969.198392:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762969.198396:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0611a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.198403:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0611a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.201695:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0611a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.201705:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0611a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.201712:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0611a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.201718:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880aa400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762969.201723:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0611a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.201729:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0611a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.201740:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0611a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.201746:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009d53932d/0x416193e012b0611a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762969.201784:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015177856:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3423us (3545us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762969.201791:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1488 00000100:00100000:3.0:1789762969.201798:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 2 deadline 492602238313 time 492502261606 state 0x2 ref 0 00000100:00100000:2.0:1789762969.203349:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762969.203399:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015177984 00000100:00100000:0.0:1789762969.203405:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015177984 00000100:00100000:0.0:1789762969.203429:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 2 deadline 492602238313 time 492502261606 state 0x0 ref 1 00000100:00100000:0.0:1789762969.203438:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 2 deadline 492602261606 time 492502261606 state 0x1 ref 1 00000100:00100000:0.0:1789762969.203463:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 1 deadline 492602261606 time 492507370286 state 0x0 ref 1 00000100:00100000:0.0:1789762969.203476:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d8257e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015177984:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762969.203495:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b06121 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.203507:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762969.203511:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762969.203515:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b06121 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.203524:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b06121 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.205838:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b06121 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.205849:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b06121 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.205855:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b06121 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.205865:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ec21d3e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762969.205869:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b06121 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.205878:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b06121 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.205890:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b06121 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.205898:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009c9a2af5/0x416193e012b06121 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [1048576->2097151] (req 1048576->2097151) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762969.205947:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d8257e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015177984:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2473us (2597us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762969.205953:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1489 00000100:00100000:0.0:1789762969.205960:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 1 deadline 492602261606 time 492507370286 state 0x2 ref 0 00000100:00100000:0.0:1789762969.207558:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762969.207594:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015178112 00000100:00100000:1.0:1789762969.207599:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015178112 00000100:00100000:1.0:1789762969.207624:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 1 deadline 492602261606 time 492507370286 state 0x0 ref 1 00000100:00100000:1.0:1789762969.207632:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 1 deadline 492607370286 time 492507370286 state 0x1 ref 1 00000100:00100000:1.0:1789762969.207651:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492607370286 time 492511561130 state 0x0 ref 1 00000100:00100000:1.0:1789762969.207660:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015178112:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762969.207677:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06128 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.207685:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762969.207688:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762969.207691:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06128 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.207699:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06128 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.209848:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06128 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.209858:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06128 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.209865:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06128 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.209870:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762969.209873:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06128 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.209878:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06128 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.209893:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06128 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.209899:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06128 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [2097152->3145727] (req 2097152->3145727) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762969.209940:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbc9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015178112:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2280us (2383us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762969.209946:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1490 00000100:00100000:1.0:1789762969.209952:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492607370286 time 492511561130 state 0x2 ref 0 00000100:00100000:3.0:1789762969.211419:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762969.211453:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015178240 00000100:00100000:1.0:1789762969.211458:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015178240 00000100:00100000:1.0:1789762969.211492:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492607370286 time 492511561130 state 0x0 ref 1 00000100:00100000:1.0:1789762969.211505:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492611561130 time 492511561130 state 0x1 ref 1 00000100:00100000:2.0:1789762969.307757:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492611561130 time 492611644631 state 0x0 ref 1 00000100:00100000:2.0:1789762969.307786:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015178240:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762969.307830:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007fadde83/0x416193e012b0612f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.307849:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762969.307853:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762969.307858:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007fadde83/0x416193e012b0612f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.307870:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007fadde83/0x416193e012b0612f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.310850:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007fadde83/0x416193e012b0612f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.310872:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007fadde83/0x416193e012b0612f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.310878:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007fadde83/0x416193e012b0612f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.310883:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f9b600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762969.310887:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007fadde83/0x416193e012b0612f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.310894:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007fadde83/0x416193e012b0612f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.310907:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007fadde83/0x416193e012b0612f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762969.310911:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007fadde83/0x416193e012b0612f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [3145728->4194303] (req 3145728->4194303) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762969.310958:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0fc7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015178240:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3184us (99539us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762969.310965:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1491 00000100:00100000:2.0:1789762969.310972:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492611561130 time 492611644631 state 0x2 ref 0 00000100:00100000:2.0:1789762969.312859:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762969.312913:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015178368 00000100:00100000:1.0:1789762969.312921:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015178368 00000100:00100000:1.0:1789762969.312958:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492611561130 time 492611644631 state 0x0 ref 1 00000100:00100000:1.0:1789762969.312969:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492711644631 time 492611644631 state 0x1 ref 1 00000100:00100000:1.0:1789762969.407790:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492711644631 time 492711687943 state 0x0 ref 1 00000100:00100000:1.0:1789762969.407810:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981d84542d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015178368:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762969.407848:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06136 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.407861:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762969.407864:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762969.407869:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06136 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.407879:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06136 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.411168:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06136 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.411179:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06136 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.411186:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06136 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.411192:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762969.411196:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06136 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.411202:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06136 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.411218:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06136 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.411224:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06136 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [4194304->5242879] (req 4194304->5242879) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762969.411272:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981d84542d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015178368:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3465us (98413us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762969.411280:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1492 00000100:00100000:1.0:1789762969.411288:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492711644631 time 492711687943 state 0x2 ref 0 00000100:00100000:0.0:1789762969.413353:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762969.413401:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015178496 00000100:00100000:3.0:1789762969.413411:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015178496 00000100:00100000:3.0:1789762969.413467:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492711644631 time 492711687943 state 0x0 ref 1 00000100:00100000:3.0:1789762969.413479:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492811687943 time 492711687943 state 0x1 ref 1 00000100:00100000:0.0:1789762969.507894:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492811687943 time 492811790353 state 0x0 ref 1 00000100:00100000:0.0:1789762969.507919:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015178496:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762969.507968:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cd76d100/0x416193e012b0613d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.507984:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762969.507987:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762969.507991:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cd76d100/0x416193e012b0613d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.508004:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cd76d100/0x416193e012b0613d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.511394:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cd76d100/0x416193e012b0613d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.511408:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cd76d100/0x416193e012b0613d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.511417:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cd76d100/0x416193e012b0613d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.511425:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e8344f800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762969.511430:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd76d100/0x416193e012b0613d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.511439:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cd76d100/0x416193e012b0613d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.511462:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cd76d100/0x416193e012b0613d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.511471:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cd76d100/0x416193e012b0613d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [5242880->6291455] (req 5242880->6291455) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762969.511558:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015178496:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3646us (98204us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762969.511570:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1493 00000100:00100000:3.0:1789762969.511581:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492811687943 time 492811790353 state 0x2 ref 0 00000100:00100000:3.0:1789762969.514595:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762969.514649:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015178624 00000100:00100000:1.0:1789762969.514659:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015178624 00000100:00100000:1.0:1789762969.514704:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492811687943 time 492811790353 state 0x0 ref 1 00000100:00100000:1.0:1789762969.514716:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492911790353 time 492811790353 state 0x1 ref 1 00010000:00010000:2.0:1789762969.527261:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762969.607979:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492911790353 time 492911874315 state 0x0 ref 1 00000100:00100000:3.0:1789762969.608003:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015178624:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762969.608050:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06144 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.608064:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762969.608067:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762969.608072:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06144 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.608083:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06144 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.611787:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06144 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.611799:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06144 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.611806:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06144 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.611812:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f18800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762969.611815:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06144 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.611830:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06144 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.611843:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06144 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.611848:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06144 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [6291456->7340031] (req 6291456->7340031) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762969.611895:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851bbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015178624:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3901us (97305us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762969.611902:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1494 00000100:00100000:3.0:1789762969.611921:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492911790353 time 492911874315 state 0x2 ref 0 00000100:00100000:3.0:1789762969.614083:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762969.614166:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015178752 00000100:00100000:1.0:1789762969.614175:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015178752 00000100:00100000:1.0:1789762969.614211:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 492911790353 time 492911874315 state 0x0 ref 1 00000100:00100000:1.0:1789762969.614222:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493011874315 time 492911874315 state 0x1 ref 1 00000100:00100000:1.0:1789762969.708044:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493011874315 time 493011938320 state 0x0 ref 1 00000100:00100000:1.0:1789762969.708084:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015178752:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762969.708154:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0614b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.708168:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762969.708172:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762969.708176:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0614b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.708189:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0614b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.711438:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0614b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.711450:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0614b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.711458:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0614b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.711464:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762969.711467:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0614b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.711475:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0614b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.711490:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0614b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.711503:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b0614b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [7340032->8388607] (req 7340032->8388607) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762969.711553:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015178752:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3479us (97470us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762969.711561:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1495 00000100:00100000:1.0:1789762969.711568:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493011874315 time 493011938320 state 0x2 ref 0 00000100:00100000:3.0:1789762969.713621:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762969.713667:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015178880 00000100:00100000:1.0:1789762969.713676:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015178880 00000100:00100000:1.0:1789762969.713714:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493011874315 time 493011938320 state 0x0 ref 1 00000100:00100000:1.0:1789762969.713725:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493111938320 time 493011938320 state 0x1 ref 1 00000100:00100000:3.0:1789762969.808148:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493111938320 time 493112044793 state 0x0 ref 1 00000100:00100000:3.0:1789762969.808175:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e851b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015178880:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762969.808211:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06152 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.808232:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762969.808236:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762969.808241:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06152 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.808253:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06152 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.811007:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06152 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.811018:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06152 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.811025:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06152 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.811030:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f18800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762969.811033:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06152 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.811040:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06152 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.811053:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06152 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762969.811059:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06152 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [8388608->9437183] (req 8388608->9437183) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762969.811140:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e851b9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015178880:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2973us (97520us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762969.811149:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1496 00000100:00100000:3.0:1789762969.811156:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493111938320 time 493112044793 state 0x2 ref 0 00000100:00100000:3.0:1789762969.813116:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762969.813159:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015179008 00000100:00100000:2.0:1789762969.813169:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015179008 00000100:00100000:2.0:1789762969.813205:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493111938320 time 493112044793 state 0x0 ref 1 00000100:00100000:2.0:1789762969.813217:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493212044793 time 493112044793 state 0x1 ref 1 00000100:00100000:1.0:1789762969.908245:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493212044793 time 493212136168 state 0x0 ref 1 00000100:00100000:1.0:1789762969.908272:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8342ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015179008:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762969.908311:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06159 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.908329:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762969.908334:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762969.908341:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06159 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762969.908356:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06159 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.911232:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06159 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.911245:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06159 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.911253:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06159 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.911259:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762969.911263:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06159 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.911269:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06159 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.911284:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06159 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762969.911294:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06159 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [9437184->10485759] (req 9437184->10485759) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762969.911344:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8342ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015179008:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3080us (98258us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762969.911353:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1497 00000100:00100000:0.0:1789762969.911361:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493212044793 time 493212136168 state 0x2 ref 0 00000100:00100000:3.0:1789762969.913302:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762969.913353:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015179136 00000100:00100000:1.0:1789762969.913361:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015179136 00000100:00100000:1.0:1789762969.913405:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493212044793 time 493212136168 state 0x0 ref 1 00000100:00100000:1.0:1789762969.913419:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493312136168 time 493212136168 state 0x1 ref 1 00000100:00100000:1.0:1789762970.008290:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493312136168 time 493312186213 state 0x0 ref 1 00000100:00100000:1.0:1789762970.008315:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8342f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015179136:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762970.008350:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06160 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.008364:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762970.008368:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762970.008372:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06160 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.008383:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06160 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.011435:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06160 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.011443:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06160 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.011451:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06160 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.011459:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762970.011464:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06160 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.011474:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06160 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.011492:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06160 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.011501:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06160 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [10485760->11534335] (req 10485760->11534335) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762970.011548:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8342f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015179136:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3243us (98248us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762970.011556:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1498 00000100:00100000:1.0:1789762970.011563:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493312136168 time 493312186213 state 0x2 ref 0 00000100:00100000:2.0:1789762970.013567:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762970.013625:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015179264 00000100:00100000:3.0:1789762970.013634:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015179264 00000100:00100000:3.0:1789762970.013678:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493312136168 time 493312186213 state 0x0 ref 1 00000100:00100000:3.0:1789762970.013692:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493412186213 time 493312186213 state 0x1 ref 1 00000100:00100000:1.0:1789762970.108367:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493412186213 time 493412262245 state 0x0 ref 1 00000100:00100000:1.0:1789762970.108388:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83422680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015179264:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762970.108422:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06167 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.108435:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762970.108439:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762970.108443:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06167 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.108456:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06167 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.111400:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06167 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.111411:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06167 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.111423:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06167 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.111429:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762970.111432:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06167 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.111439:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06167 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.111455:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06167 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.111461:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b06167 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [11534336->12582911] (req 11534336->12582911) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762970.111515:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83422680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015179264:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3132us (97946us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762970.111526:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1499 00000100:00100000:3.0:1789762970.111537:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493412186213 time 493412262245 state 0x2 ref 0 00000100:00100000:2.0:1789762970.113426:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762970.113461:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015179392 00000100:00100000:3.0:1789762970.113469:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015179392 00000100:00100000:3.0:1789762970.113503:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493412186213 time 493412262245 state 0x0 ref 1 00000100:00100000:3.0:1789762970.113517:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493512262245 time 493412262245 state 0x1 ref 1 00000100:00100000:2.0:1789762970.208472:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493512262245 time 493512368828 state 0x0 ref 1 00000100:00100000:2.0:1789762970.208494:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83421180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015179392:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762970.208529:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007f083c83/0x416193e012b0616e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.208543:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762970.208546:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762970.208551:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f083c83/0x416193e012b0616e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.208561:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f083c83/0x416193e012b0616e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.211778:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007f083c83/0x416193e012b0616e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.211793:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007f083c83/0x416193e012b0616e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.211805:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007f083c83/0x416193e012b0616e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.211813:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83450600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762970.211818:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007f083c83/0x416193e012b0616e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.211828:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007f083c83/0x416193e012b0616e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.211846:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007f083c83/0x416193e012b0616e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.211851:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f083c83/0x416193e012b0616e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [12582912->13631487] (req 12582912->13631487) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762970.211900:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83421180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015179392:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3413us (98475us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762970.211910:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1500 00000100:00100000:3.0:1789762970.211918:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493512262245 time 493512368828 state 0x2 ref 0 00000100:00100000:2.0:1789762970.213897:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762970.213931:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015179520 00000100:00100000:3.0:1789762970.213938:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015179520 00000100:00100000:3.0:1789762970.213970:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493512262245 time 493512368828 state 0x0 ref 1 00000100:00100000:3.0:1789762970.213980:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493612368828 time 493512368828 state 0x1 ref 1 00000100:00100000:1.0:1789762970.308537:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493612368828 time 493612434008 state 0x0 ref 1 00000100:00100000:1.0:1789762970.308559:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83423100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015179520:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762970.308596:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06175 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.308610:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762970.308614:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762970.308618:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06175 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.308629:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06175 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.311667:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06175 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.311678:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06175 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.311685:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06175 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.311691:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762970.311694:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06175 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.311701:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06175 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.311714:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06175 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.311719:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003cb287b8/0x416193e012b06175 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [13631488->14680063] (req 13631488->14680063) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762970.311778:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83423100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015179520:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3223us (97879us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762970.311788:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1501 00000100:00100000:1.0:1789762970.311799:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493612368828 time 493612434008 state 0x2 ref 0 00000100:00100000:2.0:1789762970.313935:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762970.313979:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015179648 00000100:00100000:3.0:1789762970.313988:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015179648 00000100:00100000:3.0:1789762970.314023:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493612368828 time 493612434008 state 0x0 ref 1 00000100:00100000:3.0:1789762970.314034:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493712434008 time 493612434008 state 0x1 ref 1 00000100:00100000:3.0:1789762970.408650:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493712434008 time 493712536575 state 0x0 ref 1 00000100:00100000:3.0:1789762970.408683:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83420e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015179648:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762970.408733:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0617c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.408755:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.408762:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.408772:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0617c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.408812:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0617c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.412216:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0617c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.412227:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0617c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.412234:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0617c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.412238:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f18800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762970.412242:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0617c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.412248:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0617c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.412261:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0617c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.412266:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0617c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [14680064->15728639] (req 14680064->15728639) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762970.412322:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83420e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015179648:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3652us (98387us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762970.412330:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1502 00000100:00100000:3.0:1789762970.412337:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493712434008 time 493712536575 state 0x2 ref 0 00000100:00100000:2.0:1789762970.414582:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762970.414636:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015179776 00000100:00100000:1.0:1789762970.414645:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015179776 00000100:00100000:1.0:1789762970.414691:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493712434008 time 493712536575 state 0x0 ref 1 00000100:00100000:1.0:1789762970.414706:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493812536575 time 493712536575 state 0x1 ref 1 00000100:00100000:3.0:1789762970.508751:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493812536575 time 493812644317 state 0x0 ref 1 00000100:00100000:3.0:1789762970.508779:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83420a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015179776:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762970.508813:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06183 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.508835:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.508839:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.508843:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06183 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.508854:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06183 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.511878:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06183 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.511888:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06183 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.511900:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06183 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.511905:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f18800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762970.511909:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06183 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.511915:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06183 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.511928:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06183 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.511933:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06183 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [15728640->16777215] (req 15728640->16777215) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762970.511979:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83420a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015179776:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3210us (97397us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762970.511989:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1503 00000100:00100000:3.0:1789762970.511997:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493812536575 time 493812644317 state 0x2 ref 0 00000100:00100000:2.0:1789762970.513905:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762970.513948:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015179904 00000100:00100000:3.0:1789762970.513957:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015179904 00000100:00100000:3.0:1789762970.513990:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493812536575 time 493812644317 state 0x0 ref 1 00000100:00100000:3.0:1789762970.514005:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493912644317 time 493812644317 state 0x1 ref 1 00010000:00010000:0.0:1789762970.551193:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762970.608811:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493912644317 time 493912706345 state 0x0 ref 1 00000100:00100000:3.0:1789762970.608837:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83420000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015179904:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762970.608890:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0618a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.608904:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.608908:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.608912:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0618a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.608923:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0618a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.611736:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0618a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.611750:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0618a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.611759:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0618a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.611773:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f18800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762970.611778:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0618a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.611788:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0618a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.611806:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0618a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.611814:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b0618a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [16777216->17825791] (req 16777216->17825791) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762970.611872:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83420000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015179904:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3045us (97967us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762970.611882:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1504 00000100:00100000:3.0:1789762970.611892:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493912644317 time 493912706345 state 0x2 ref 0 00000100:00100000:2.0:1789762970.614256:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762970.614304:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015180032 00000100:00100000:0.0:1789762970.614312:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015180032 00000100:00100000:0.0:1789762970.614351:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 493912644317 time 493912706345 state 0x0 ref 1 00000100:00100000:0.0:1789762970.614362:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494012706345 time 493912706345 state 0x1 ref 1 00000100:00100000:3.0:1789762970.708972:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494012706345 time 494012861660 state 0x0 ref 1 00000100:00100000:3.0:1789762970.709006:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83421500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015180032:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762970.709056:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06191 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.709150:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762970.709159:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.709169:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06191 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.709195:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06191 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.714488:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06191 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.714509:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06191 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.714523:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06191 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.714534:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f18800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762970.714540:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06191 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.714552:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06191 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.714574:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06191 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.714611:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f1d74d25/0x416193e012b06191 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [17825792->18874367] (req 17825792->18874367) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762970.714728:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83421500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015180032:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5730us (100466us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762970.714747:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1505 00000100:00100000:3.0:1789762970.714764:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494012706345 time 494012861660 state 0x2 ref 0 00000100:00100000:2.0:1789762970.718861:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762970.718955:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015180160 00000100:00100000:1.0:1789762970.718970:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015180160 00000100:00100000:1.0:1789762970.719029:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494012706345 time 494012861660 state 0x0 ref 1 00000100:00100000:1.0:1789762970.719048:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494112861660 time 494012861660 state 0x1 ref 1 00000004:00080000:0.0:1789762970.757688:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 34 committed 00000001:00080000:0.0:1789762970.757729:0:5826:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967646 is committed 00000004:00080000:0.0:1789762970.757747:0:5826:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 27 committed 00000100:00100000:1.0:1789762970.757892:0:3271:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789762970.757923:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e8342ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022202880:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1789762970.757955:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e83423480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022203008:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1789762970.758001:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762970.758072:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762970.758169:0:6649:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022202880 00000100:00100000:0.0:1789762970.758180:0:6649:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022202880 00000100:00100000:2.0:1789762970.758210:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022203008 00000100:00100000:2.0:1789762970.758225:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022203008 00000100:00080000:1.0:1789762970.758246:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789762970.758256:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789762970.758294:0:6649:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:3.0:1789762970.758319:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 231 00000100:00100000:0.0:1789762970.758325:0:6649:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022202880:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1789762970.758346:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8342e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3273:x1876702022203008:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1789762970.758361:0:6649:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:25 count 1 00002000:00080000:3.0:1789762970.758380:0:6881:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:16 count 1 00002000:00010000:0.0:1789762970.758406:0:6649:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x19:0x0] initial LVB size: 8388608, mtime: 0x6aad9d93, atime: 0x0, ctime: 0x6aad9d93, blocks: 0x4007 00002000:00010000:3.0:1789762970.758418:0:6881:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x10:0x0] initial LVB size: 8388608, mtime: 0x6aad9d93, atime: 0x0, ctime: 0x6aad9d93, blocks: 0x4007 00010000:00010000:0.0:1789762970.758421:0:6649:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebf9c20b/0x416193e012b06198 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.758438:0:6881:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f1d74d25/0x416193e012b0619f lrc: 3/0,1 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762970.758447:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=494s v=(0 0) 00010000:00010000:0.0:1789762970.758463:0:6649:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.758465:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=494s v=(0 0) 00010000:00010000:0.0:1789762970.758473:0:6649:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ebf9c20b/0x416193e012b06198 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.758476:0:6881:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.758498:0:6881:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f1d74d25/0x416193e012b0619f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762970.758505:0:6649:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ebf9c20b/0x416193e012b06198 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762970.758548:0:6649:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000ebf9c20b/0x416193e012b06198 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.758556:0:6881:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f1d74d25/0x416193e012b0619f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762970.758579:0:6649:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ebf9c20b/0x416193e012b06198 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.758637:0:6881:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f1d74d25/0x416193e012b0619f lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762970.758655:0:6649:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789762970.758661:0:6881:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f1d74d25/0x416193e012b0619f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.758684:0:8466:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789762970.758712:0:6881:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762970.758721:0:8466:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ebf9c20b/0x416193e012b06198 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.758748:0:8466:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87925800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762970.758755:0:8466:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebf9c20b/0x416193e012b06198 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.758769:0:8466:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ebf9c20b/0x416193e012b06198 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.758792:0:8466:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ebf9c20b/0x416193e012b06198 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.758803:0:8466:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ebf9c20b/0x416193e012b06198 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6649 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.758828:0:6663:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762970.758832:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762970.758841:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762970.758847:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762970.758849:0:6663:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f1d74d25/0x416193e012b0619f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.758874:0:6663:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f18800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762970.758885:0:6663:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f1d74d25/0x416193e012b0619f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.758901:0:6663:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f1d74d25/0x416193e012b0619f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.758925:0:6663:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f1d74d25/0x416193e012b0619f lrc: 1/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.758937:0:6663:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f1d74d25/0x416193e012b0619f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6881 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762970.758968:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762970.758971:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762970.758977:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762970.759904:0:6649:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022202880:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1588us (1899us total) trans 4294968369 rc 0/0 00000100:00100000:0.0:1789762970.759925:0:6649:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000004:00080000:1.0:1789762970.759960:0:3271:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981e8342ea00/2, rc 0, transno 4294968369 00000100:00100000:1.0:1789762970.759975:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e8342ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022202880:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1789762970.760257:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8342e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3273:x1876702022203008:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1920us (2190us total) trans 4294967452 rc 0/0 00000100:00100000:3.0:1789762970.760270:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 231 00000100:00100000:2.0:1789762970.760323:0:3273:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967451 gen 2 00000004:00080000:2.0:1789762970.760334:0:3273:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981d81394000, transno 4294967451 00000100:00100000:2.0:1789762970.760362:0:3273:0:(client.c:3169:ptlrpc_free_committed()) @@@ stopping search req@ffff981e83423480 x1876702022203008/t4294967452(4294967452) o6->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789762986 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1789762970.760397:0:3273:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff981e83423480/2, rc 0, transno 4294967452 00000100:00100000:2.0:1789762970.760404:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e83423480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3273:1876702022203008:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1789762970.809051:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494112861660 time 494112942822 state 0x0 ref 1 00000100:00100000:1.0:1789762970.809081:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83422300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015180160:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762970.809178:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061a6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.809205:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762970.809211:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762970.809218:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061a6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.809233:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061a6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.814494:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061a6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.814516:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061a6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.814528:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061a6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.814539:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83451c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762970.814545:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061a6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.814556:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061a6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.814577:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061a6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762970.814597:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061a6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [18874368->19922943] (req 18874368->19922943) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762970.814675:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83422300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015180160:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5601us (95815us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762970.814689:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1506 00000100:00100000:1.0:1789762970.814703:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494112861660 time 494112942822 state 0x2 ref 0 00000100:00100000:1.0:1789762970.818593:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762970.818680:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015180288 00000100:00100000:3.0:1789762970.818693:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015180288 00000100:00100000:3.0:1789762970.818751:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494112861660 time 494112942822 state 0x0 ref 1 00000100:00100000:3.0:1789762970.818770:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494212942822 time 494112942822 state 0x1 ref 1 00000100:00100000:3.0:1789762970.909195:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494212942822 time 494213086768 state 0x0 ref 1 00000100:00100000:3.0:1789762970.909225:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015180288:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762970.909286:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ad lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.909307:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.909312:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762970.909320:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ad lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.909336:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ad lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.912799:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ad lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.912809:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ad lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.912816:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.912821:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762970.912825:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.912831:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ad lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.912845:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ad lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762970.912850:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ad lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [19922944->20971519] (req 19922944->20971519) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762970.912900:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015180288:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3689us (94313us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762970.912908:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1507 00000100:00100000:3.0:1789762970.912916:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494212942822 time 494213086768 state 0x2 ref 0 00000100:00100000:1.0:1789762970.915158:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762970.915216:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015180416 00000100:00100000:2.0:1789762970.915243:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015180416 00000100:00100000:2.0:1789762970.915306:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494212942822 time 494213086768 state 0x0 ref 1 00000100:00100000:2.0:1789762970.915342:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494313086768 time 494213086768 state 0x1 ref 1 00000100:00100000:0.0:1789762971.009383:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494313086768 time 494313271539 state 0x0 ref 1 00000100:00100000:0.0:1789762971.009459:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015180416:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762971.009563:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b061b4 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789762971.009564:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789762971.009590:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762971.009599:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762971.009610:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b061b4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762971.009635:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b061b4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762971.014250:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b061b4 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762971.014275:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b061b4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762971.014290:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b061b4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762971.014300:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe18600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762971.014306:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b061b4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762971.014318:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b061b4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762971.014338:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b061b4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762971.014347:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b061b4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [20971520->22020095] (req 20971520->22020095) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762971.014421:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015180416:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4998us (99261us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762971.014433:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1508 00000100:00100000:0.0:1789762971.014445:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494313086768 time 494313271539 state 0x2 ref 0 00000100:00100000:2.0:1789762971.017778:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762971.017867:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015180544 00000100:00100000:1.0:1789762971.017878:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015180544 00000100:00100000:1.0:1789762971.017932:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494313086768 time 494313271539 state 0x0 ref 1 00000100:00100000:1.0:1789762971.017966:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494413271539 time 494313271539 state 0x1 ref 1 00000100:00100000:1.0:1789762971.109432:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494413271539 time 494413328245 state 0x0 ref 1 00000100:00100000:1.0:1789762971.109456:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83420700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015180544:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762971.109493:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061bb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.109508:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762971.109512:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762971.109517:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.109536:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.113033:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.113045:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061bb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.113053:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061bb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.113058:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83451c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762971.113061:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061bb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.113067:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061bb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.113080:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061bb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.113115:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb083358/0x416193e012b061bb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [22020096->23068671] (req 22020096->23068671) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762971.113166:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83420700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015180544:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3718us (95391us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762971.113174:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1509 00000100:00100000:1.0:1789762971.113182:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494413271539 time 494413328245 state 0x2 ref 0 00000100:00100000:0.0:1789762971.115197:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762971.115240:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015180672 00000100:00100000:3.0:1789762971.115248:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015180672 00000100:00100000:3.0:1789762971.115286:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494413271539 time 494413328245 state 0x0 ref 1 00000100:00100000:3.0:1789762971.115296:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494513328245 time 494413328245 state 0x1 ref 1 00000100:00100000:3.0:1789762971.209494:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494513328245 time 494513385233 state 0x0 ref 1 00000100:00100000:3.0:1789762971.209519:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015180672:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762971.209555:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061c2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.209568:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762971.209571:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762971.209576:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061c2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.209586:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061c2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.216312:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061c2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.216325:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061c2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.216333:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061c2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.216338:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762971.216343:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061c2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.216353:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061c2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.216374:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061c2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.216382:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061c2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [23068672->24117247] (req 23068672->24117247) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762971.216443:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015180672:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6932us (101244us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762971.216454:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1510 00000100:00100000:3.0:1789762971.216466:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494513328245 time 494513385233 state 0x2 ref 0 00000100:00100000:0.0:1789762971.218461:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762971.218513:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015180800 00000100:00100000:1.0:1789762971.218524:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015180800 00000100:00100000:1.0:1789762971.218573:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494513328245 time 494513385233 state 0x0 ref 1 00000100:00100000:1.0:1789762971.218589:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494613385233 time 494513385233 state 0x1 ref 1 00000100:00100000:2.0:1789762971.309573:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494613385233 time 494613468900 state 0x0 ref 1 00000100:00100000:2.0:1789762971.309598:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015180800:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762971.309642:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005f4f07fc/0x416193e012b061c9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762971.309656:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762971.309660:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762971.309665:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005f4f07fc/0x416193e012b061c9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762971.309676:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005f4f07fc/0x416193e012b061c9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762971.312218:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005f4f07fc/0x416193e012b061c9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762971.312242:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005f4f07fc/0x416193e012b061c9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762971.312253:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005f4f07fc/0x416193e012b061c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762971.312261:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d8291f000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762971.312266:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005f4f07fc/0x416193e012b061c9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762971.312275:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005f4f07fc/0x416193e012b061c9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762971.312293:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005f4f07fc/0x416193e012b061c9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762971.312301:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005f4f07fc/0x416193e012b061c9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [24117248->25165823] (req 24117248->25165823) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762971.312361:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbc8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015180800:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2772us (93899us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762971.312372:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1511 00000100:00100000:2.0:1789762971.312382:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494613385233 time 494613468900 state 0x2 ref 0 00000100:00100000:2.0:1789762971.314321:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762971.314366:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015180928 00000100:00100000:3.0:1789762971.314375:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015180928 00000100:00100000:3.0:1789762971.314410:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494613385233 time 494613468900 state 0x0 ref 1 00000100:00100000:3.0:1789762971.314420:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494713468900 time 494613468900 state 0x1 ref 1 00000100:00100000:1.0:1789762971.409652:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494713468900 time 494713549549 state 0x0 ref 1 00000100:00100000:1.0:1789762971.409674:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83421c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015180928:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762971.409705:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2db2c6c/0x416193e012b061d0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.409717:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762971.409720:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762971.409725:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2db2c6c/0x416193e012b061d0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.409737:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2db2c6c/0x416193e012b061d0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.412733:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2db2c6c/0x416193e012b061d0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.412745:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2db2c6c/0x416193e012b061d0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.412751:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2db2c6c/0x416193e012b061d0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.412757:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981d83568c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762971.412761:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2db2c6c/0x416193e012b061d0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.412767:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2db2c6c/0x416193e012b061d0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.412781:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2db2c6c/0x416193e012b061d0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.412790:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2db2c6c/0x416193e012b061d0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [25165824->26214399] (req 25165824->26214399) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762971.412837:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83421c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015180928:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3170us (98517us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762971.412844:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1512 00000100:00100000:3.0:1789762971.412851:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494713468900 time 494713549549 state 0x2 ref 0 00000100:00100000:2.0:1789762971.414968:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762971.415003:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015181056 00000100:00100000:3.0:1789762971.415011:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015181056 00000100:00100000:3.0:1789762971.415041:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494713468900 time 494713549549 state 0x0 ref 1 00000100:00100000:3.0:1789762971.415051:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494813549549 time 494713549549 state 0x1 ref 1 00000100:00100000:3.0:1789762971.509716:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494813549549 time 494813612182 state 0x0 ref 1 00000100:00100000:3.0:1789762971.509739:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83421f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015181056:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762971.509772:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061d7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.509785:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762971.509789:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762971.509794:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061d7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.509808:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061d7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.513177:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061d7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.513188:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061d7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.513195:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061d7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.513200:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762971.513203:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061d7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.513209:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061d7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.513222:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061d7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.513228:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061d7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [26214400->27262975] (req 26214400->27262975) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762971.513278:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83421f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015181056:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3547us (98311us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762971.513284:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1513 00000100:00100000:3.0:1789762971.513291:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494813549549 time 494813612182 state 0x2 ref 0 00000100:00100000:2.0:1789762971.515207:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762971.515257:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015181184 00000100:00100000:1.0:1789762971.515265:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015181184 00000100:00100000:1.0:1789762971.515301:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494813549549 time 494813612182 state 0x0 ref 1 00000100:00100000:1.0:1789762971.515312:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494913612182 time 494813612182 state 0x1 ref 1 00010000:00010000:2.0:1789762971.575238:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762971.609767:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494913612182 time 494913664825 state 0x0 ref 1 00000100:00100000:1.0:1789762971.609792:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83421880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015181184:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762971.609828:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061de lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.609843:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762971.609846:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762971.609850:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061de lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.609860:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061de lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.612827:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061de lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.612838:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061de lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.612846:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061de lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.612852:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9955b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762971.612856:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061de lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.612862:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061de lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.612877:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061de lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.612883:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061de lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [27262976->28311551] (req 27262976->28311551) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762971.612931:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83421880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015181184:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3149us (97725us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762971.612938:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1514 00000100:00100000:1.0:1789762971.612945:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494913612182 time 494913664825 state 0x2 ref 0 00000100:00100000:1.0:1789762971.615158:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762971.615191:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015181312 00000100:00100000:3.0:1789762971.615200:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015181312 00000100:00100000:3.0:1789762971.615236:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 494913612182 time 494913664825 state 0x0 ref 1 00000100:00100000:3.0:1789762971.615246:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495013664825 time 494913664825 state 0x1 ref 1 00000100:00100000:3.0:1789762971.709836:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495013664825 time 495013729045 state 0x0 ref 1 00000100:00100000:3.0:1789762971.709862:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4971880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015181312:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762971.709897:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061e5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.709911:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762971.709914:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762971.709919:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061e5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.709931:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061e5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.712524:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061e5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.712535:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061e5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.712542:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061e5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.712548:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762971.712551:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061e5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.712557:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061e5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.712570:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061e5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.712576:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061e5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [28311552->29360127] (req 28311552->29360127) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762971.712622:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4971880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015181312:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2771us (97464us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762971.712628:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1515 00000100:00100000:3.0:1789762971.712636:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495013664825 time 495013729045 state 0x2 ref 0 00000100:00100000:0.0:1789762971.714632:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762971.714668:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015181440 00000100:00100000:3.0:1789762971.714676:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015181440 00000100:00100000:3.0:1789762971.714711:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495013664825 time 495013729045 state 0x0 ref 1 00000100:00100000:3.0:1789762971.714721:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495113729045 time 495013729045 state 0x1 ref 1 00000100:00100000:1.0:1789762971.746124:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762971.746153:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762971.746158:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015181568 00000100:00100000:0.0:1789762971.746165:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015181568 00000100:00100000:1.0:1789762971.746185:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762971.746207:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015181696 00000100:00100000:0.0:1789762971.746213:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 366 00000100:00100000:3.0:1789762971.746215:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015181696 00000100:00100000:1.0:1789762971.746222:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762971.746226:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015181568:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762971.746247:0:6647:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015181824 00000100:00100000:2.0:1789762971.746253:0:6647:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015181824 00000100:00100000:1.0:1789762971.746258:0:8165:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015181952 00000100:00100000:1.0:1789762971.746264:0:8165:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015181952 00000100:00100000:0.0:1789762971.746277:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2366:x1876702015181568:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 54us (193us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762971.746282:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 402 00000100:00100000:0.0:1789762971.746288:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 366 00000100:00100000:2.0:1789762971.746296:0:6647:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 199 00000100:00100000:3.0:1789762971.746297:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4970380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702015181696:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762971.746307:0:6647:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4970e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015181824:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762971.746310:0:8165:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 403 00000100:00100000:1.0:1789762971.746324:0:8165:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4973800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015181952:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:0.0:1789762971.746334:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789762971.746341:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762971.746344:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4970380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2365:x1876702015181696:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 49us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762971.746350:0:6647:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4970e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2366:x1876702015181824:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 46us (166us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762971.746355:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 402 00000100:00100000:2.0:1789762971.746359:0:6647:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 199 00000100:00100000:1.0:1789762971.746363:0:8165:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4973800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2365:x1876702015181952:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 40us (141us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762971.746372:0:8165:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 403 00000100:00100000:1.0:1789762971.767235:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e83420380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022203136:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789762971.767277:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762971.767318:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022203136 00000100:00100000:0.0:1789762971.767324:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022203136 00000100:00100000:0.0:1789762971.767358:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:0.0:1789762971.767366:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4972a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022203136:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762971.767389:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3746562048 avail 3744464896 00000100:00100000:0.0:1789762971.767430:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4972a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3274:x1876702022203136:12345-0@lo:13:osp-pre-0-0.0 Request processed in 65us (151us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762971.767435:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00000100:00100000:1.0:1789762971.767471:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e83420380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022203136:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762971.809893:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495113729045 time 495113790849 state 0x0 ref 1 00000100:00100000:3.0:1789762971.809915:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015181440:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762971.809946:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ec lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.809959:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762971.809962:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762971.809967:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.809982:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.813163:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.813177:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ec lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.813188:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.813196:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762971.813200:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.813210:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ec lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.813227:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ec lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.813236:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061ec lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [29360128->30408703] (req 29360128->30408703) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762971.813292:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbc9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015181440:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3384us (98660us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762971.813299:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1516 00000100:00100000:3.0:1789762971.813306:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495113729045 time 495113790849 state 0x2 ref 0 00000100:00100000:1.0:1789762971.815312:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762971.815339:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015182080 00000100:00100000:3.0:1789762971.815347:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015182080 00000100:00100000:3.0:1789762971.815378:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495113729045 time 495113790849 state 0x0 ref 1 00000100:00100000:3.0:1789762971.815388:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495213790849 time 495113790849 state 0x1 ref 1 00000100:00100000:1.0:1789762971.909980:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495213790849 time 495213875957 state 0x0 ref 1 00000100:00100000:1.0:1789762971.910004:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981eb4972300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015182080:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762971.910046:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061f3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.910066:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762971.910070:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762971.910075:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061f3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762971.910125:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061f3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.913625:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061f3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.913641:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061f3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.913651:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.913660:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9955b000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762971.913665:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.913685:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061f3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.913704:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061f3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762971.913713:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077b181e4/0x416193e012b061f3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [30408704->31457279] (req 30408704->31457279) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762971.913778:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981eb4972300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015182080:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3782us (98466us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762971.913786:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1517 00000100:00100000:3.0:1789762971.913794:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495213790849 time 495213875957 state 0x2 ref 0 00000100:00100000:0.0:1789762971.915921:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762971.915954:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015182208 00000100:00100000:3.0:1789762971.915961:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015182208 00000100:00100000:3.0:1789762971.916009:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495213790849 time 495213875957 state 0x0 ref 1 00000100:00100000:3.0:1789762971.916020:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495313875957 time 495213875957 state 0x1 ref 1 00000100:00100000:3.0:1789762972.010047:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495313875957 time 495313939589 state 0x0 ref 1 00000100:00100000:3.0:1789762972.010078:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015182208:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762972.010165:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061fa lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.010188:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.010193:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.010198:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061fa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.010210:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061fa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.012963:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061fa lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.012979:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061fa lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.012990:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061fa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.012997:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762972.013003:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061fa lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.013012:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061fa lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.013041:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061fa lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.013050:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b061fa lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [31457280->32505855] (req 31457280->32505855) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762972.013156:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcdf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015182208:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3088us (97233us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762972.013184:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1518 00000100:00100000:3.0:1789762972.013195:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495313875957 time 495313939589 state 0x2 ref 0 00000100:00100000:0.0:1789762972.015488:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762972.015555:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015182336 00000100:00100000:2.0:1789762972.015566:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015182336 00000100:00100000:2.0:1789762972.015625:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495313875957 time 495313939589 state 0x0 ref 1 00000100:00100000:2.0:1789762972.015642:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495413939589 time 495313939589 state 0x1 ref 1 00000100:00100000:3.0:1789762972.110171:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495413939589 time 495414066735 state 0x0 ref 1 00000100:00100000:3.0:1789762972.110194:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015182336:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762972.110232:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06201 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.110247:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.110250:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.110255:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06201 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.110266:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06201 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.113551:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06201 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.113560:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06201 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.113569:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06201 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.113574:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762972.113577:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06201 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.113583:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06201 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.113595:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06201 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.113600:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06201 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [32505856->33554431] (req 32505856->33554431) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762972.113645:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015182336:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3460us (98162us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762972.113653:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1519 00000100:00100000:3.0:1789762972.113661:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495413939589 time 495414066735 state 0x2 ref 0 00000100:00100000:2.0:1789762972.115657:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762972.115702:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015182464 00000100:00100000:1.0:1789762972.115710:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015182464 00000100:00100000:1.0:1789762972.115746:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495413939589 time 495414066735 state 0x0 ref 1 00000100:00100000:1.0:1789762972.115768:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495514066735 time 495414066735 state 0x1 ref 1 00000100:00100000:3.0:1789762972.210277:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495514066735 time 495514173994 state 0x0 ref 1 00000100:00100000:3.0:1789762972.210300:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83423b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015182464:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762972.210335:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06208 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.210349:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.210353:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.210357:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06208 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.210368:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06208 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.213316:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06208 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.213327:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06208 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.213334:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06208 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.213340:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762972.213344:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06208 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.213351:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06208 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.213368:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06208 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.213373:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06208 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [33554432->34603007] (req 33554432->34603007) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762972.213421:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83423b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015182464:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3130us (97764us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762972.213437:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1520 00000100:00100000:3.0:1789762972.213445:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495514066735 time 495514173994 state 0x2 ref 0 00000100:00100000:1.0:1789762972.215362:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762972.215414:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015182592 00000100:00100000:2.0:1789762972.215435:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015182592 00000100:00100000:2.0:1789762972.215473:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495514066735 time 495514173994 state 0x0 ref 1 00000100:00100000:2.0:1789762972.215484:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495614173994 time 495514173994 state 0x1 ref 1 00000100:00100000:1.0:1789762972.279258:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfbcc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022203392:0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.279258:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfbcfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022203520:0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.279277:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfbced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022203648:0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.279293:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e8342d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022203264:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762972.279345:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762972.279348:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762972.279351:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789762972.279391:0:5798:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022203392 00000100:00100000:2.0:1789762972.279400:0:5798:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022203392 00000100:00100000:0.0:1789762972.279405:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfbcf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022203776:0@lo:400:kworker.0 00000100:00100000:1.0:1789762972.279415:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfbcea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022203904:0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.279427:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762972.279455:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762972.279458:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022203520 00000100:00100000:0.0:1789762972.279463:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022203520 00000100:00100000:2.0:1789762972.279472:0:5798:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 367 00000100:00100000:3.0:1789762972.279490:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022203264 00000100:00100000:2.0:1789762972.279493:0:5798:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85086c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022203392:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.279496:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022203264 00000100:00100000:1.0:1789762972.279502:0:6648:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022203904 00000100:00100000:0.0:1789762972.279503:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022203648 00000100:00100000:1.0:1789762972.279507:0:6648:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022203904 00000100:00100000:0.0:1789762972.279508:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022203648 00000100:00100000:0.0:1789762972.279551:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:2.0:1789762972.279561:0:5798:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85086c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3272:x1876702022203392:12345-0@lo:400:kworker.0 Request processed in 77us (217us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.279565:0:6648:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:0.0:1789762972.279567:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8342f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022203520:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.279568:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:2.0:1789762972.279588:0:5798:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 367 00000100:00100000:1.0:1789762972.279593:0:6648:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3272:x1876702022203904:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.279594:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3271:x1876702022203264:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789762972.279626:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3851419648 avail 3849322496 00000100:00100000:0.0:1789762972.279633:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8342f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3273:x1876702022203520:12345-0@lo:400:kworker.0 Request processed in 73us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762972.279641:0:6648:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3272:x1876702022203904:12345-0@lo:400:kworker.0 Request processed in 63us (188us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762972.279643:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:1.0:1789762972.279653:0:6648:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00100000:2.0:1789762972.279661:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:3.0:1789762972.279675:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbce680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3271:x1876702022203264:12345-0@lo:13:osp-pre-1-0.0 Request processed in 93us (326us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762972.279676:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8342ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022203648:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.279690:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:1.0:1789762972.279698:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfbcea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3272:1876702022203904:0@lo:400:kworker.0 00000100:00100000:0.0:1789762972.279700:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfbcfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3273:1876702022203520:0@lo:400:kworker.0 00000100:00100000:1.0:1789762972.279719:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfbcc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3272:1876702022203392:0@lo:400:kworker.0 00000100:00100000:2.0:1789762972.279741:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8342ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3274:x1876702022203648:12345-0@lo:400:kworker.0 Request processed in 71us (314us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762972.279743:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789762972.279749:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762972.279750:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:0.0:1789762972.279754:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762972.279778:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022203776 00000100:00100000:3.0:1789762972.279783:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022203776 00000100:00100000:0.0:1789762972.279813:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e8342d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022203264:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789762972.279821:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfbced80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3274:1876702022203648:0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.279829:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:3.0:1789762972.279840:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022203776:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789762972.279857:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762972.279878:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3271:x1876702022203776:12345-0@lo:400:kworker.0 Request processed in 42us (131us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762972.279884:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:0.0:1789762972.279911:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfbcf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3271:1876702022203776:0@lo:400:kworker.0 00000100:00100000:3.0:1789762972.310369:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495614173994 time 495614264262 state 0x0 ref 1 00000100:00100000:3.0:1789762972.310395:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015182592:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762972.310431:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0620f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.310445:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.310449:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.310453:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0620f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.310464:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0620f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.314365:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0620f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.314375:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0620f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.314382:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0620f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.314388:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762972.314392:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0620f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.314398:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0620f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.314410:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0620f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.314415:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0620f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [34603008->35651583] (req 34603008->35651583) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762972.314463:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015182592:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4079us (99101us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762972.314472:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1521 00000100:00100000:3.0:1789762972.314493:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495614173994 time 495614264262 state 0x2 ref 0 00000100:00100000:0.0:1789762972.316934:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762972.317010:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015182720 00000100:00100000:1.0:1789762972.317018:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015182720 00000100:00100000:1.0:1789762972.317056:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495614173994 time 495614264262 state 0x0 ref 1 00000100:00100000:1.0:1789762972.317069:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495714264262 time 495614264262 state 0x1 ref 1 00000100:00100000:2.0:1789762972.410453:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495714264262 time 495714349690 state 0x0 ref 1 00000100:00100000:2.0:1789762972.410480:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015182720:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762972.410514:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06216 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.410527:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762972.410531:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762972.410537:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06216 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.410552:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06216 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.414253:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06216 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.414265:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06216 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.414273:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06216 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.414279:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfb2d800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762972.414282:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06216 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.414289:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06216 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.414302:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06216 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.414307:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06216 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [35651584->36700159] (req 35651584->36700159) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762972.414355:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015182720:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3881us (97421us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762972.414361:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1522 00000100:00100000:2.0:1789762972.414376:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495714264262 time 495714349690 state 0x2 ref 0 00000100:00100000:2.0:1789762972.416591:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762972.416638:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015182848 00000100:00100000:3.0:1789762972.416646:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015182848 00000100:00100000:3.0:1789762972.416695:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495714264262 time 495714349690 state 0x0 ref 1 00000100:00100000:3.0:1789762972.416706:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495814349690 time 495714349690 state 0x1 ref 1 00000100:00100000:1.0:1789762972.510552:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495814349690 time 495814448197 state 0x0 ref 1 00000100:00100000:1.0:1789762972.510575:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83422d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015182848:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762972.510612:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0621d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.510626:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762972.510629:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762972.510634:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0621d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.510644:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0621d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.513380:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0621d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.513391:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0621d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.513399:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0621d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.513409:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762972.513413:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0621d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.513419:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0621d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.513432:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0621d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.513437:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0621d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [36700160->37748735] (req 36700160->37748735) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.513483:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83422d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015182848:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2916us (96891us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762972.513489:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1523 00000100:00100000:1.0:1789762972.513497:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495814349690 time 495814448197 state 0x2 ref 0 00000100:00100000:1.0:1789762972.515546:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762972.515587:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015182976 00000100:00100000:3.0:1789762972.515597:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015182976 00000100:00100000:3.0:1789762972.515646:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495814349690 time 495814448197 state 0x0 ref 1 00000100:00100000:3.0:1789762972.515662:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495914448197 time 495814448197 state 0x1 ref 1 00010000:00010000:0.0:1789762972.599228:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762972.610619:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495914448197 time 495914518405 state 0x0 ref 1 00000100:00100000:2.0:1789762972.610642:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015182976:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762972.610678:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06224 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.610692:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762972.610696:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762972.610700:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06224 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.610711:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06224 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.613707:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06224 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.613719:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06224 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.613726:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06224 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.613744:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfb2d800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762972.613749:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06224 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.613755:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06224 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.613770:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06224 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762972.613775:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000007a01bb3/0x416193e012b06224 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [37748736->38797311] (req 37748736->38797311) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762972.613823:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015182976:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3188us (98276us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762972.613830:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1524 00000100:00100000:2.0:1789762972.613837:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495914448197 time 495914518405 state 0x2 ref 0 00000100:00100000:0.0:1789762972.615785:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762972.615829:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015183104 00000100:00100000:3.0:1789762972.615838:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015183104 00000100:00100000:3.0:1789762972.615877:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 495914448197 time 495914518405 state 0x0 ref 1 00000100:00100000:3.0:1789762972.615887:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496014518405 time 495914518405 state 0x1 ref 1 00000100:00100000:1.0:1789762972.710697:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496014518405 time 496014593327 state 0x0 ref 1 00000100:00100000:1.0:1789762972.710720:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015183104:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762972.710755:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0622b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.710770:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762972.710774:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762972.710778:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0622b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.710792:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0622b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.713659:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0622b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.713670:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0622b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.713677:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0622b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.713682:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f01800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762972.713686:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0622b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.713692:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0622b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.713705:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0622b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762972.713714:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000037bd77c7/0x416193e012b0622b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [38797312->39845887] (req 38797312->39845887) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762972.713763:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015183104:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3051us (97979us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762972.713771:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1525 00000100:00100000:1.0:1789762972.713779:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496014518405 time 496014593327 state 0x2 ref 0 00000100:00100000:2.0:1789762972.715605:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762972.715643:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015183232 00000100:00100000:3.0:1789762972.715652:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015183232 00000100:00100000:3.0:1789762972.715690:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496014518405 time 496014593327 state 0x0 ref 1 00000100:00100000:3.0:1789762972.715701:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496114593327 time 496014593327 state 0x1 ref 1 00000100:00100000:3.0:1789762972.810746:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496114593327 time 496114644442 state 0x0 ref 1 00000100:00100000:3.0:1789762972.810768:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83423800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015183232:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762972.810801:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06232 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.810813:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.810816:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.810821:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06232 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.810833:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06232 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.813302:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06232 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.813314:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06232 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.813324:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06232 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.813331:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762972.813336:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06232 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.813345:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06232 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.813363:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06232 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.813371:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06232 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [39845888->40894463] (req 39845888->40894463) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762972.813428:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83423800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015183232:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2666us (97822us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762972.813438:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1526 00000100:00100000:3.0:1789762972.813448:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496114593327 time 496114644442 state 0x2 ref 0 00000100:00100000:0.0:1789762972.815322:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762972.815360:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015183360 00000100:00100000:3.0:1789762972.815367:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015183360 00000100:00100000:3.0:1789762972.815398:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496114593327 time 496114644442 state 0x0 ref 1 00000100:00100000:3.0:1789762972.815409:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496214644442 time 496114644442 state 0x1 ref 1 00000100:00100000:3.0:1789762972.910815:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496214644442 time 496214711769 state 0x0 ref 1 00000100:00100000:3.0:1789762972.910841:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015183360:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762972.910877:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06239 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.910891:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.910894:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762972.910899:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06239 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.910910:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06239 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.914029:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06239 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.914038:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06239 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.914045:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06239 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.914049:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762972.914053:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06239 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.914059:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06239 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.914071:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06239 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762972.914076:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06239 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [40894464->41943039] (req 40894464->41943039) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762972.914148:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015183360:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3316us (98827us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762972.914157:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1527 00000100:00100000:3.0:1789762972.914168:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496214644442 time 496214711769 state 0x2 ref 0 00000100:00100000:0.0:1789762972.916184:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762972.916229:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015183488 00000100:00100000:1.0:1789762972.916237:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015183488 00000100:00100000:1.0:1789762972.916273:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496214644442 time 496214711769 state 0x0 ref 1 00000100:00100000:1.0:1789762972.916283:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496314711769 time 496214711769 state 0x1 ref 1 00000100:00100000:3.0:1789762973.010897:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496314711769 time 496314792064 state 0x0 ref 1 00000100:00100000:3.0:1789762973.010934:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015183488:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762973.010970:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06240 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.010985:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.010989:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.010994:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06240 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762973.011000:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762973.011006:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06240 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.013956:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06240 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.013966:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06240 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.013973:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06240 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.013978:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762973.013983:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06240 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.013990:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06240 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.014005:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06240 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.014010:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06240 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [41943040->42991615] (req 41943040->42991615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762973.014055:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015183488:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3142us (97871us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762973.014063:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1528 00000100:00100000:3.0:1789762973.014071:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496314711769 time 496314792064 state 0x2 ref 0 00000100:00100000:2.0:1789762973.015869:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762973.015922:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015183616 00000100:00100000:1.0:1789762973.015931:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015183616 00000100:00100000:1.0:1789762973.015974:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496314711769 time 496314792064 state 0x0 ref 1 00000100:00100000:1.0:1789762973.015989:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496414792064 time 496314792064 state 0x1 ref 1 00000100:00100000:3.0:1789762973.111025:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496414792064 time 496414907592 state 0x0 ref 1 00000100:00100000:3.0:1789762973.111054:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83422a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015183616:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762973.111140:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06247 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.111155:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.111158:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.111163:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06247 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.111175:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06247 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.113994:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06247 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.114006:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06247 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.114013:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06247 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.114017:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762973.114021:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06247 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.114027:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06247 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.114051:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06247 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.114057:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06247 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [42991616->44040191] (req 42991616->44040191) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762973.114168:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83422a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015183616:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3123us (98296us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762973.114180:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1529 00000100:00100000:3.0:1789762973.114191:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496414792064 time 496414907592 state 0x2 ref 0 00000100:00100000:2.0:1789762973.116271:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762973.116304:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015183744 00000100:00100000:3.0:1789762973.116311:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015183744 00000100:00100000:3.0:1789762973.116347:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496414792064 time 496414907592 state 0x0 ref 1 00000100:00100000:3.0:1789762973.116357:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496514907592 time 496414907592 state 0x1 ref 1 00000100:00100000:3.0:1789762973.211126:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496514907592 time 496514973004 state 0x0 ref 1 00000100:00100000:3.0:1789762973.211151:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83420380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015183744:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762973.211191:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0624e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.211208:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.211212:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.211217:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0624e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.211228:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0624e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.214791:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0624e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.214801:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0624e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.214809:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0624e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.214815:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762973.214819:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0624e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.214825:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0624e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.214840:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0624e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.214846:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0624e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [44040192->45088767] (req 44040192->45088767) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762973.214894:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83420380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015183744:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3753us (98624us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762973.214900:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1530 00000100:00100000:3.0:1789762973.214907:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496514907592 time 496514973004 state 0x2 ref 0 00000100:00100000:0.0:1789762973.217186:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762973.217236:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015183872 00000100:00100000:1.0:1789762973.217245:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015183872 00000100:00100000:1.0:1789762973.217283:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496514907592 time 496514973004 state 0x0 ref 1 00000100:00100000:1.0:1789762973.217298:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496614973004 time 496514973004 state 0x1 ref 1 00000100:00100000:1.0:1789762973.311160:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496614973004 time 496615056662 state 0x0 ref 1 00000100:00100000:1.0:1789762973.311182:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015183872:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762973.311229:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b06255 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.311244:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762973.311247:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762973.311252:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b06255 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.311262:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b06255 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.314479:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b06255 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.314490:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b06255 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.314496:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b06255 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.314502:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe18600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762973.314505:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b06255 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.314512:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b06255 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.314525:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b06255 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.314530:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b06255 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [45088768->46137343] (req 45088768->46137343) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762973.314576:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015183872:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3402us (97391us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762973.314583:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1531 00000100:00100000:1.0:1789762973.314591:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496614973004 time 496615056662 state 0x2 ref 0 00000100:00100000:1.0:1789762973.316651:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762973.316692:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015184000 00000100:00100000:2.0:1789762973.316700:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015184000 00000100:00100000:2.0:1789762973.316746:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496614973004 time 496615056662 state 0x0 ref 1 00000100:00100000:2.0:1789762973.316765:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496715056662 time 496615056662 state 0x1 ref 1 00000100:00100000:1.0:1789762973.411237:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496715056662 time 496715133919 state 0x0 ref 1 00000100:00100000:1.0:1789762973.411257:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015184000:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762973.411291:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0625c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.411305:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762973.411308:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762973.411313:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0625c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.411324:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0625c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.413949:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0625c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.413961:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0625c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.413968:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0625c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.413974:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe18600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762973.413978:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0625c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.413984:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0625c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.413999:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0625c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.414004:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0625c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [46137344->47185919] (req 46137344->47185919) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762973.414066:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015184000:12345-192.168.203.48@tcp:3:dd.500 Request processed in 2814us (97415us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762973.414134:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1532 00000100:00100000:1.0:1789762973.414146:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496715056662 time 496715133919 state 0x2 ref 0 00000100:00100000:1.0:1789762973.416009:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762973.416050:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015184128 00000100:00100000:3.0:1789762973.416059:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015184128 00000100:00100000:3.0:1789762973.416130:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496715056662 time 496715133919 state 0x0 ref 1 00000100:00100000:3.0:1789762973.416141:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496815133919 time 496715133919 state 0x1 ref 1 00000100:00100000:3.0:1789762973.511317:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496815133919 time 496815212374 state 0x0 ref 1 00000100:00100000:3.0:1789762973.511343:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015184128:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762973.511378:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06263 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.511392:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.511396:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.511403:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06263 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.511418:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06263 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.515230:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06263 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.515244:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06263 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.515255:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06263 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.515262:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762973.515266:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06263 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.515273:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06263 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.515286:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06263 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.515292:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06263 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [47185920->48234495] (req 47185920->48234495) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762973.515338:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015184128:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4007us (99330us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762973.515346:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1533 00000100:00100000:3.0:1789762973.515353:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496815133919 time 496815212374 state 0x2 ref 0 00000100:00100000:1.0:1789762973.517429:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762973.517500:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015184256 00000100:00100000:0.0:1789762973.517511:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015184256 00000100:00100000:0.0:1789762973.517561:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496815133919 time 496815212374 state 0x0 ref 1 00000100:00100000:0.0:1789762973.517573:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496915212374 time 496815212374 state 0x1 ref 1 00000100:00100000:1.0:1789762973.611417:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496915212374 time 496915304274 state 0x0 ref 1 00000100:00100000:1.0:1789762973.611441:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015184256:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762973.611477:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0626a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.611492:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762973.611504:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762973.611511:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0626a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762973.611528:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0626a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.614793:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0626a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.614805:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0626a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.614813:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0626a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.614818:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfe18600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762973.614822:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0626a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.614828:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0626a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.614843:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0626a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.614848:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001ef7680b/0x416193e012b0626a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [48234496->49283071] (req 48234496->49283071) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762973.614897:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015184256:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3463us (97467us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762973.614904:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1534 00000100:00100000:3.0:1789762973.614912:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496915212374 time 496915304274 state 0x2 ref 0 00000100:00100000:2.0:1789762973.616896:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762973.616927:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015184384 00000100:00100000:3.0:1789762973.616934:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015184384 00000100:00100000:3.0:1789762973.616971:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 496915212374 time 496915304274 state 0x0 ref 1 00000100:00100000:3.0:1789762973.616985:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497015304274 time 496915304274 state 0x1 ref 1 00010000:00010000:1.0:1789762973.623204:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789762973.711522:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497015304274 time 497015417474 state 0x0 ref 1 00000100:00100000:2.0:1789762973.711548:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015184384:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762973.711585:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077de9f1e/0x416193e012b06271 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.711599:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762973.711602:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762973.711607:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077de9f1e/0x416193e012b06271 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762973.711620:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077de9f1e/0x416193e012b06271 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.715025:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000077de9f1e/0x416193e012b06271 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.715048:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077de9f1e/0x416193e012b06271 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.715056:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077de9f1e/0x416193e012b06271 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.715061:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfcd7000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762973.715064:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077de9f1e/0x416193e012b06271 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.715070:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077de9f1e/0x416193e012b06271 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.715127:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077de9f1e/0x416193e012b06271 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.715133:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077de9f1e/0x416193e012b06271 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [49283072->50331647] (req 49283072->50331647) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762973.715180:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015184384:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3643us (98284us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762973.715190:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1535 00000100:00100000:3.0:1789762973.715203:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497015304274 time 497015417474 state 0x2 ref 0 00000100:00100000:0.0:1789762973.717243:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762973.717274:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015184512 00000100:00100000:3.0:1789762973.717282:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015184512 00000100:00100000:3.0:1789762973.717315:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497015304274 time 497015417474 state 0x0 ref 1 00000100:00100000:3.0:1789762973.717325:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497115417474 time 497015417474 state 0x1 ref 1 00000100:00100000:3.0:1789762973.811568:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497115417474 time 497115466487 state 0x0 ref 1 00000100:00100000:3.0:1789762973.811593:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015184512:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762973.811624:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06278 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.811636:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.811640:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762973.811646:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06278 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.811656:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06278 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.814736:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06278 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.814750:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06278 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.814756:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06278 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.814762:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762973.814766:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06278 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.814772:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06278 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.814791:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06278 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762973.814800:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06278 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [50331648->51380223] (req 50331648->51380223) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762973.814850:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015184512:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3263us (97607us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762973.814858:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1536 00000100:00100000:3.0:1789762973.814865:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497115417474 time 497115466487 state 0x2 ref 0 00000100:00100000:1.0:1789762973.816812:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762973.816849:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015184640 00000100:00100000:2.0:1789762973.816858:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015184640 00000100:00100000:2.0:1789762973.816902:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497115417474 time 497115466487 state 0x0 ref 1 00000100:00100000:2.0:1789762973.816917:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497215466487 time 497115466487 state 0x1 ref 1 00000001:00080000:3.0:1789762973.824228:0:7605:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967452 is committed 00000100:00100000:0.0:1789762973.911684:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497215466487 time 497215579929 state 0x0 ref 1 00000100:00100000:0.0:1789762973.911706:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015184640:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762973.911742:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0627f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.911755:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762973.911758:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762973.911763:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0627f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.911775:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0627f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.915906:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0627f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.915916:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0627f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.915923:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0627f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.915929:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfcd3400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762973.915932:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0627f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.915938:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0627f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.915961:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0627f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762973.915966:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0627f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [51380224->52428799] (req 51380224->52428799) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762973.916014:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015184640:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4315us (99204us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762973.916021:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1537 00000100:00100000:0.0:1789762973.916028:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497215466487 time 497215579929 state 0x2 ref 0 00000100:00100000:1.0:1789762973.918496:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762973.918596:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015184768 00000100:00100000:3.0:1789762973.918616:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015184768 00000100:00100000:3.0:1789762973.918701:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497215466487 time 497215579929 state 0x0 ref 1 00000100:00100000:3.0:1789762973.918721:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497315579929 time 497215579929 state 0x1 ref 1 00000100:00100000:3.0:1789762974.011788:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497315579929 time 497315666935 state 0x0 ref 1 00000100:00100000:3.0:1789762974.011830:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015184768:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762974.011889:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06286 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.011910:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.011915:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.011922:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06286 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.011938:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06286 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.016356:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06286 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.016370:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06286 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.016380:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06286 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.016391:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762974.016397:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06286 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.016408:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06286 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.016428:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06286 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.016437:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06286 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [52428800->53477375] (req 52428800->53477375) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762974.016521:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015184768:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4704us (98021us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762974.016537:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1538 00000100:00100000:3.0:1789762974.016553:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497315579929 time 497315666935 state 0x2 ref 0 00000100:00100000:1.0:1789762974.019059:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762974.019172:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015184896 00000100:00100000:3.0:1789762974.019186:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015184896 00000100:00100000:3.0:1789762974.019261:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497315579929 time 497315666935 state 0x0 ref 1 00000100:00100000:3.0:1789762974.019279:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497415666935 time 497315666935 state 0x1 ref 1 00000100:00100000:3.0:1789762974.111836:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497415666935 time 497415725722 state 0x0 ref 1 00000100:00100000:3.0:1789762974.111872:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015184896:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762974.111963:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0628d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.111986:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.112001:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.112009:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0628d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.112028:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0628d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.116005:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0628d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.116025:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0628d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.116039:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0628d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.116050:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762974.116058:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0628d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.116122:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0628d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.116150:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0628d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.116162:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0628d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [53477376->54525951] (req 53477376->54525951) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762974.116251:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015184896:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4393us (97205us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762974.116267:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1539 00000100:00100000:3.0:1789762974.116282:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497415666935 time 497415725722 state 0x2 ref 0 00000100:00100000:2.0:1789762974.118592:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762974.118663:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015185024 00000100:00100000:1.0:1789762974.118679:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015185024 00000100:00100000:1.0:1789762974.118731:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497415666935 time 497415725722 state 0x0 ref 1 00000100:00100000:1.0:1789762974.118755:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497515725722 time 497415725722 state 0x1 ref 1 00000100:00100000:2.0:1789762974.211967:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497515725722 time 497515831253 state 0x0 ref 1 00000100:00100000:2.0:1789762974.212034:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f29880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015185024:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762974.212198:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06294 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.212241:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762974.212248:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762974.212257:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06294 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.212280:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06294 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.217495:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06294 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.217513:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06294 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.217526:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06294 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.217537:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762974.217544:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06294 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.217590:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06294 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.217616:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06294 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762974.217625:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b06294 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [54525952->55574527] (req 54525952->55574527) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762974.217733:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f29880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015185024:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5709us (99138us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762974.217763:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1540 00000100:00100000:2.0:1789762974.217775:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497515725722 time 497515831253 state 0x2 ref 0 00000100:00100000:2.0:1789762974.221334:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762974.221434:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015185152 00000100:00100000:0.0:1789762974.221468:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015185152 00000100:00100000:0.0:1789762974.221555:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497515725722 time 497515831253 state 0x0 ref 1 00000100:00100000:0.0:1789762974.221577:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497615831253 time 497515831253 state 0x1 ref 1 00000100:00100000:0.0:1789762974.311976:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497615831253 time 497615874879 state 0x0 ref 1 00000100:00100000:0.0:1789762974.311996:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f28380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015185152:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762974.312042:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0629b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.312056:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762974.312059:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762974.312064:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0629b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.312131:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0629b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.317602:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0629b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.317615:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0629b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.317623:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0629b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.317630:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ebfcd3400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762974.317636:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0629b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.317643:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0629b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.317674:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0629b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.317680:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a2dba564/0x416193e012b0629b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [55574528->56623103] (req 55574528->56623103) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762974.317778:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f28380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2366:x1876702015185152:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5785us (96448us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762974.317792:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1541 00000100:00100000:1.0:1789762974.317802:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497615831253 time 497615874879 state 0x2 ref 0 00000100:00100000:1.0:1789762974.321625:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762974.321688:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015185280 00000100:00100000:1.0:1789762974.321696:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015185280 00000100:00100000:1.0:1789762974.321741:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497615831253 time 497615874879 state 0x0 ref 1 00000100:00100000:1.0:1789762974.321772:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497715874879 time 497615874879 state 0x1 ref 1 00000001:00080000:0.0:1789762974.362609:0:6674:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294968369 is committed 00000100:00100000:3.0:1789762974.412188:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497715874879 time 497716078267 state 0x0 ref 1 00000100:00100000:3.0:1789762974.412221:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015185280:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762974.412281:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062a2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.412304:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.412311:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.412318:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062a2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.412341:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062a2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.416645:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062a2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.416662:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062a2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.416673:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062a2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.416682:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762974.416689:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062a2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.416698:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062a2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.416720:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062a2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.416728:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062a2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [56623104->57671679] (req 56623104->57671679) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762974.416807:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015185280:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4597us (95181us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762974.416819:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1542 00000100:00100000:3.0:1789762974.416831:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497715874879 time 497716078267 state 0x2 ref 0 00000100:00100000:2.0:1789762974.420243:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762974.420291:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015185408 00000100:00100000:3.0:1789762974.420303:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015185408 00000100:00100000:3.0:1789762974.420356:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497715874879 time 497716078267 state 0x0 ref 1 00000100:00100000:3.0:1789762974.420372:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497816078267 time 497716078267 state 0x1 ref 1 00000100:00100000:1.0:1789762974.512322:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497816078267 time 497816218131 state 0x0 ref 1 00000100:00100000:1.0:1789762974.512347:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f2bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015185408:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762974.512387:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062a9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.512405:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762974.512410:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762974.512417:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.512433:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.516152:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.516166:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062a9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.516176:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062a9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.516184:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83450e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762974.516189:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062a9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.516199:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062a9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.516217:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062a9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762974.516226:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062a9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [57671680->58720255] (req 57671680->58720255) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762974.516282:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f2bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015185408:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3944us (96042us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762974.516288:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1543 00000100:00100000:1.0:1789762974.516296:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497816078267 time 497816218131 state 0x2 ref 0 00000100:00100000:0.0:1789762974.519195:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762974.519264:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015185536 00000100:00100000:3.0:1789762974.519282:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015185536 00000100:00100000:3.0:1789762974.519356:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497816078267 time 497816218131 state 0x0 ref 1 00000100:00100000:3.0:1789762974.519374:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497916218131 time 497816218131 state 0x1 ref 1 00000100:00100000:0.0:1789762974.612486:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497916218131 time 497916380671 state 0x0 ref 1 00000100:00100000:0.0:1789762974.612514:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015185536:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762974.612560:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000641fcff/0x416193e012b062b0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.612576:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762974.612580:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762974.612585:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000641fcff/0x416193e012b062b0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.612602:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000641fcff/0x416193e012b062b0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.617650:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000641fcff/0x416193e012b062b0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.617661:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000641fcff/0x416193e012b062b0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.617669:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000641fcff/0x416193e012b062b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.617675:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f31200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762974.617679:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000641fcff/0x416193e012b062b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.617686:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000641fcff/0x416193e012b062b0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.617702:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000641fcff/0x416193e012b062b0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762974.617707:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000641fcff/0x416193e012b062b0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [58720256->59768831] (req 58720256->59768831) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762974.617771:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015185536:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5269us (98576us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762974.617779:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1544 00000100:00100000:0.0:1789762974.617787:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497916218131 time 497916380671 state 0x2 ref 0 00000100:00100000:0.0:1789762974.620944:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762974.621025:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015185664 00000100:00100000:3.0:1789762974.621045:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015185664 00000100:00100000:3.0:1789762974.621221:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 497916218131 time 497916380671 state 0x0 ref 1 00000100:00100000:3.0:1789762974.621243:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498016380671 time 497916380671 state 0x1 ref 1 00010000:00010000:2.0:1789762974.647250:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789762974.712603:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498016380671 time 498016487757 state 0x0 ref 1 00000100:00100000:3.0:1789762974.712649:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015185664:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762974.712727:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062b7 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.712760:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789762974.712767:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.712776:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062b7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.712801:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062b7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.719145:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062b7 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.719170:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062b7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.719179:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062b7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.719185:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762974.719189:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062b7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.719201:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062b7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.719227:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062b7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.719238:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062b7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [59768832->60817407] (req 59768832->60817407) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762974.719311:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbce300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015185664:12345-192.168.203.48@tcp:3:dd.500 Request processed in 6680us (98365us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762974.719320:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1545 00000100:00100000:3.0:1789762974.719330:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498016380671 time 498016487757 state 0x2 ref 0 00000100:00100000:0.0:1789762974.722570:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762974.722654:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015185792 00000100:00100000:1.0:1789762974.722665:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015185792 00000100:00100000:1.0:1789762974.722713:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498016380671 time 498016487757 state 0x0 ref 1 00000100:00100000:1.0:1789762974.722724:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498116487757 time 498016487757 state 0x1 ref 1 00000100:00100000:3.0:1789762974.812660:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498116487757 time 498116555489 state 0x0 ref 1 00000100:00100000:3.0:1789762974.812683:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015185792:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762974.812720:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062be lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.812735:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.812738:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.812743:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062be lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.812754:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062be lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.817662:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062be lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.817671:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062be lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.817678:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062be lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.817685:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762974.817689:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062be lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.817696:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062be lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.817715:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062be lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.817721:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062be lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [60817408->61865983] (req 60817408->61865983) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762974.817784:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015185792:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5109us (95214us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762974.817794:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1546 00000100:00100000:3.0:1789762974.817802:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498116487757 time 498116555489 state 0x2 ref 0 00000100:00100000:0.0:1789762974.820902:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762974.820976:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015185920 00000100:00100000:1.0:1789762974.820984:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015185920 00000100:00100000:1.0:1789762974.821028:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498116487757 time 498116555489 state 0x0 ref 1 00000100:00100000:1.0:1789762974.821044:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498216555489 time 498116555489 state 0x1 ref 1 00000100:00100000:3.0:1789762974.912748:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498216555489 time 498216643801 state 0x0 ref 1 00000100:00100000:3.0:1789762974.912773:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015185920:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762974.912812:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062c5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.912826:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.912830:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762974.912834:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062c5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.912846:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062c5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.917684:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062c5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.917697:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062c5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.917706:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062c5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.917713:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762974.917718:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062c5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.917727:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062c5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.917744:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062c5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762974.917752:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062c5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [61865984->62914559] (req 61865984->62914559) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762974.917808:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015185920:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5043us (96905us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762974.917815:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1547 00000100:00100000:3.0:1789762974.917821:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498216555489 time 498216643801 state 0x2 ref 0 00000100:00100000:0.0:1789762974.921186:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762974.921294:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015186048 00000100:00100000:2.0:1789762974.921315:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015186048 00000100:00100000:2.0:1789762974.921400:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498216555489 time 498216643801 state 0x0 ref 1 00000100:00100000:2.0:1789762974.921423:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498316643801 time 498216643801 state 0x1 ref 1 00000100:00100000:3.0:1789762975.012928:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498316643801 time 498316811140 state 0x0 ref 1 00000100:00100000:3.0:1789762975.012990:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015186048:12345-192.168.203.48@tcp:3:dd.500 00000100:00080000:2.0:1789762975.013030:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762975.013058:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062cc lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.013131:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762975.013138:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762975.013147:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062cc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.013169:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062cc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.018558:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062cc lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.018579:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062cc lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.018592:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062cc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.018602:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762975.018609:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062cc lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.018622:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062cc lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.018656:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062cc lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.018672:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062cc lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [62914560->63963135] (req 62914560->63963135) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762975.018816:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015186048:12345-192.168.203.48@tcp:3:dd.500 Request processed in 5856us (97621us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762975.018837:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1548 00000100:00100000:3.0:1789762975.018855:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498316643801 time 498316811140 state 0x2 ref 0 00000100:00100000:0.0:1789762975.022932:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762975.023008:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015186176 00000100:00100000:1.0:1789762975.023022:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015186176 00000100:00100000:1.0:1789762975.023139:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498316643801 time 498316811140 state 0x0 ref 1 00000100:00100000:1.0:1789762975.023159:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498416811140 time 498316811140 state 0x1 ref 1 00000100:00100000:1.0:1789762975.113022:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498416811140 time 498416901173 state 0x0 ref 1 00000100:00100000:1.0:1789762975.113135:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015186176:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762975.113188:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062d3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.113210:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762975.113215:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762975.113221:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062d3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.113236:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062d3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.116477:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062d3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.116486:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062d3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.116493:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062d3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.116498:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83450e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762975.116502:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062d3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.116510:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062d3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.116526:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062d3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.116537:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062d3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [63963136->65011711] (req 63963136->65011711) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762975.116595:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbced80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015186176:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3546us (93671us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762975.116602:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1549 00000100:00100000:1.0:1789762975.116611:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498416811140 time 498416901173 state 0x2 ref 0 00000100:00100000:0.0:1789762975.118886:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762975.118937:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015186304 00000100:00100000:3.0:1789762975.118951:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015186304 00000100:00100000:3.0:1789762975.119003:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498416811140 time 498416901173 state 0x0 ref 1 00000100:00100000:3.0:1789762975.119015:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498516901173 time 498416901173 state 0x1 ref 1 00000100:00100000:3.0:1789762975.213084:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498516901173 time 498516962409 state 0x0 ref 1 00000100:00100000:3.0:1789762975.213142:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015186304:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762975.213186:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062da lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.213204:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762975.213210:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762975.213215:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062da lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.213227:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062da lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.216241:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062da lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.216252:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062da lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.216259:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062da lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.216268:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762975.216274:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062da lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.216280:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062da lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.216301:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062da lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.216307:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062da lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [65011712->66060287] (req 65011712->66060287) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762975.216368:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2365:x1876702015186304:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3244us (97480us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762975.216377:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1550 00000100:00100000:3.0:1789762975.216390:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498516901173 time 498516962409 state 0x2 ref 0 00000100:00100000:2.0:1789762975.219233:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762975.219342:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015186432 00000100:00100000:1.0:1789762975.219361:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015186432 00000100:00100000:1.0:1789762975.219444:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498516901173 time 498516962409 state 0x0 ref 1 00000100:00100000:1.0:1789762975.219466:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498616962409 time 498516962409 state 0x1 ref 1 00000100:00100000:2.0:1789762975.313242:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498616962409 time 498617137773 state 0x0 ref 1 00000100:00100000:2.0:1789762975.313267:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f2ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015186432:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762975.313304:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b9f4d892/0x416193e012b062e1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.313318:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762975.313321:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762975.313326:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b9f4d892/0x416193e012b062e1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.313337:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b9f4d892/0x416193e012b062e1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.317141:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b9f4d892/0x416193e012b062e1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.317157:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b9f4d892/0x416193e012b062e1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.317167:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b9f4d892/0x416193e012b062e1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.317177:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87ca0c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762975.317182:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b9f4d892/0x416193e012b062e1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.317209:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b9f4d892/0x416193e012b062e1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.317223:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b9f4d892/0x416193e012b062e1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762975.317229:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b9f4d892/0x416193e012b062e1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [66060288->67108863] (req 66060288->67108863) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762975.317284:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f2ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015186432:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4026us (98051us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762975.317290:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1551 00000100:00100000:2.0:1789762975.317297:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498616962409 time 498617137773 state 0x2 ref 0 00000100:00100000:2.0:1789762975.320808:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762975.320902:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015186560 00000100:00100000:0.0:1789762975.320922:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015186560 00000100:00100000:0.0:1789762975.321003:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498616962409 time 498617137773 state 0x0 ref 1 00000100:00100000:0.0:1789762975.321024:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498717137773 time 498617137773 state 0x1 ref 1 00000100:00100000:1.0:1789762975.413446:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498717137773 time 498717330033 state 0x0 ref 1 00000100:00100000:1.0:1789762975.413484:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f2b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015186560:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762975.413548:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062e8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.413575:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762975.413582:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762975.413591:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062e8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.413620:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062e8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.417421:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062e8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.417443:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062e8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.417457:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.417469:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83450e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762975.417476:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.417488:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062e8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.417514:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062e8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.417525:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062e8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [67108864->68157439] (req 67108864->68157439) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762975.417625:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f2b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015186560:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4152us (96813us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762975.417638:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1552 00000100:00100000:1.0:1789762975.417653:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498717137773 time 498717330033 state 0x2 ref 0 00000100:00100000:2.0:1789762975.421345:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762975.421390:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015186688 00000100:00100000:3.0:1789762975.421400:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015186688 00000100:00100000:3.0:1789762975.421449:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498717137773 time 498717330033 state 0x0 ref 1 00000100:00100000:3.0:1789762975.421461:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498817330033 time 498717330033 state 0x1 ref 1 00000100:00100000:3.0:1789762975.513488:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498817330033 time 498817383649 state 0x0 ref 1 00000100:00100000:3.0:1789762975.513511:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015186688:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762975.513548:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062ef lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.513562:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762975.513568:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762975.513582:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062ef lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.513599:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062ef lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.517188:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062ef lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.517197:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062ef lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.517204:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062ef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.517210:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762975.517213:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062ef lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.517220:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062ef lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.517233:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062ef lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.517237:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b062ef lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [68157440->69206015] (req 68157440->69206015) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762975.517288:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f2aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015186688:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3784us (95950us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762975.517295:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1553 00000100:00100000:3.0:1789762975.517302:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498817330033 time 498817383649 state 0x2 ref 0 00000100:00100000:2.0:1789762975.519848:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762975.519886:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015186816 00000100:00100000:3.0:1789762975.519894:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015186816 00000100:00100000:3.0:1789762975.519926:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498817330033 time 498817383649 state 0x0 ref 1 00000100:00100000:3.0:1789762975.519940:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498917383649 time 498817383649 state 0x1 ref 1 00000100:00100000:1.0:1789762975.613656:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498917383649 time 498917534557 state 0x0 ref 1 00000100:00100000:1.0:1789762975.613706:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f2a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015186816:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762975.613773:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062f6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.613796:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762975.613803:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762975.613810:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062f6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.613828:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062f6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.617523:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062f6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.617545:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062f6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.617571:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062f6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.617585:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83450e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762975.617594:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062f6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.617611:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062f6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.617639:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062f6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762975.617672:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b062f6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [69206016->70254591] (req 69206016->70254591) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762975.617794:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f2a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015186816:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4107us (97939us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762975.617815:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1554 00000100:00100000:1.0:1789762975.617836:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498917383649 time 498917534557 state 0x2 ref 0 00000100:00100000:0.0:1789762975.620759:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762975.620824:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015186944 00000100:00100000:3.0:1789762975.620836:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015186944 00000100:00100000:3.0:1789762975.620895:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 498917383649 time 498917534557 state 0x0 ref 1 00000100:00100000:3.0:1789762975.620908:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499017534557 time 498917534557 state 0x1 ref 1 00010000:00010000:2.0:1789762975.671339:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789762975.713806:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499017534557 time 499017676574 state 0x0 ref 1 00000100:00100000:0.0:1789762975.713846:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfbcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015186944:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762975.713916:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c19383d5/0x416193e012b062fd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762975.713963:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789762975.713970:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762975.713978:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c19383d5/0x416193e012b062fd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762975.713999:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c19383d5/0x416193e012b062fd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762975.717853:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c19383d5/0x416193e012b062fd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762975.717875:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c19383d5/0x416193e012b062fd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762975.717894:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c19383d5/0x416193e012b062fd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762975.717911:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87ca0200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762975.717922:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c19383d5/0x416193e012b062fd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762975.717942:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c19383d5/0x416193e012b062fd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762975.717969:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c19383d5/0x416193e012b062fd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762975.717988:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c19383d5/0x416193e012b062fd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [70254592->71303167] (req 70254592->71303167) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762975.718159:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfbcea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015186944:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4327us (97403us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762975.718171:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1555 00000100:00100000:0.0:1789762975.718183:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499017534557 time 499017676574 state 0x2 ref 0 00000100:00100000:0.0:1789762975.720694:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762975.720756:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015187072 00000100:00100000:3.0:1789762975.720768:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015187072 00000100:00100000:3.0:1789762975.720819:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499017534557 time 499017676574 state 0x0 ref 1 00000100:00100000:3.0:1789762975.720831:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499117676574 time 499017676574 state 0x1 ref 1 00000100:00100000:3.0:1789762975.813841:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499117676574 time 499117729806 state 0x0 ref 1 00000100:00100000:3.0:1789762975.813865:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb32300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015187072:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762975.813909:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06304 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.813923:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762975.813927:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762975.813932:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06304 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.813946:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06304 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.817075:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06304 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.817117:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06304 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.817126:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06304 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.817132:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762975.817136:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06304 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.817145:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06304 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.817162:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06304 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.817168:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06304 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [71303168->72351743] (req 71303168->72351743) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762975.817238:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb32300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015187072:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3382us (96549us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762975.817245:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1556 00000100:00100000:3.0:1789762975.817253:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499117676574 time 499117729806 state 0x2 ref 0 00000100:00100000:0.0:1789762975.820008:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762975.820148:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015187200 00000100:00100000:1.0:1789762975.820168:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015187200 00000100:00100000:1.0:1789762975.820267:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499117676574 time 499117729806 state 0x0 ref 1 00000100:00100000:1.0:1789762975.820308:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499217729806 time 499117729806 state 0x1 ref 1 00000100:00100000:3.0:1789762975.914034:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499217729806 time 499217905705 state 0x0 ref 1 00000100:00100000:3.0:1789762975.914140:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015187200:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762975.914205:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0630b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.914229:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762975.914236:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762975.914244:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0630b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.914263:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0630b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.918469:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0630b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.918484:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0630b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.918504:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0630b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.918513:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762975.918519:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0630b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.918530:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0630b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.918557:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0630b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762975.918571:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0630b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [72351744->73400319] (req 72351744->73400319) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762975.918689:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015187200:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4628us (98689us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762975.918707:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1557 00000100:00100000:3.0:1789762975.918741:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499217729806 time 499217905705 state 0x2 ref 0 00000100:00100000:0.0:1789762975.921650:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762975.921749:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015187328 00000100:00100000:1.0:1789762975.921764:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015187328 00000100:00100000:1.0:1789762975.921824:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499217729806 time 499217905705 state 0x0 ref 1 00000100:00100000:1.0:1789762975.921843:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499317905705 time 499217905705 state 0x1 ref 1 00000100:00100000:3.0:1789762976.014212:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499317905705 time 499318093150 state 0x0 ref 1 00000100:00100000:3.0:1789762976.014257:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015187328:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762976.014325:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06312 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.014351:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762976.014358:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762976.014367:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06312 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.014388:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06312 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.018720:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06312 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.018749:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06312 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.018771:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06312 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.018789:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762976.018798:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06312 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.018818:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06312 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.018854:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06312 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.018865:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06312 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [73400320->74448895] (req 73400320->74448895) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762976.018957:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015187328:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4717us (97309us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762976.018971:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1558 00000100:00100000:3.0:1789762976.018985:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499317905705 time 499318093150 state 0x2 ref 0 00000100:00100000:0.0:1789762976.022260:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762976.022380:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015187456 00000100:00100000:2.0:1789762976.022398:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015187456 00000100:00100000:2.0:1789762976.022480:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499317905705 time 499318093150 state 0x0 ref 1 00000100:00100000:2.0:1789762976.022502:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499418093150 time 499318093150 state 0x1 ref 1 00000100:00100000:3.0:1789762976.114333:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499418093150 time 499418216358 state 0x0 ref 1 00000100:00100000:3.0:1789762976.114370:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015187456:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762976.114447:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06319 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.114511:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762976.114517:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762976.114526:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06319 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.114546:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06319 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.118359:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06319 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.118376:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06319 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.118403:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06319 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.118410:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762976.118415:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06319 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.118425:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06319 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.118444:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06319 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.118471:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06319 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [74448896->75497471] (req 74448896->75497471) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762976.118560:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015187456:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4202us (96306us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762976.118571:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1559 00000100:00100000:3.0:1789762976.118585:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499418093150 time 499418216358 state 0x2 ref 0 00000100:00100000:0.0:1789762976.121332:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762976.121419:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015187584 00000100:00100000:1.0:1789762976.121435:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015187584 00000100:00100000:1.0:1789762976.121519:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499418093150 time 499418216358 state 0x0 ref 1 00000100:00100000:1.0:1789762976.121542:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499518216358 time 499418216358 state 0x1 ref 1 00000100:00100000:3.0:1789762976.214476:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499518216358 time 499518361772 state 0x0 ref 1 00000100:00100000:3.0:1789762976.214508:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015187584:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762976.214566:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06320 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.214586:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762976.214591:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762976.214598:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06320 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.214613:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06320 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.218186:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06320 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.218200:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06320 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.218210:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06320 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.218217:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762976.218224:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06320 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.218234:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06320 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.218253:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06320 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.218260:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06320 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [75497472->76546047] (req 75497472->76546047) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762976.218324:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015187584:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3828us (96995us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762976.218335:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1560 00000100:00100000:3.0:1789762976.218345:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499518216358 time 499518361772 state 0x2 ref 0 00000100:00100000:2.0:1789762976.220762:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762976.220905:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015187712 00000100:00100000:1.0:1789762976.220921:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015187712 00000100:00100000:1.0:1789762976.220997:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499518216358 time 499518361772 state 0x0 ref 1 00000100:00100000:1.0:1789762976.221012:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499618361772 time 499518361772 state 0x1 ref 1 00000100:00100000:1.0:1789762976.314589:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499618361772 time 499618466049 state 0x0 ref 1 00000100:00100000:1.0:1789762976.314633:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f29180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015187712:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762976.314692:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b06327 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.314730:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762976.314737:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762976.314745:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b06327 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.314764:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b06327 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.318811:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b06327 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.318832:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b06327 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.318849:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b06327 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.318863:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83450e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762976.318872:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b06327 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.318886:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b06327 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.318908:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b06327 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.318918:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000076bd4e22/0x416193e012b06327 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [76546048->77594623] (req 76546048->77594623) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762976.319024:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f29180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015187712:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4405us (98262us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762976.319036:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1561 00000100:00100000:3.0:1789762976.319051:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499618361772 time 499618466049 state 0x2 ref 0 00000100:00100000:2.0:1789762976.321854:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762976.321945:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015187840 00000100:00100000:3.0:1789762976.321960:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015187840 00000100:00100000:3.0:1789762976.322023:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499618361772 time 499618466049 state 0x0 ref 1 00000100:00100000:3.0:1789762976.322044:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499718466049 time 499618466049 state 0x1 ref 1 00000100:00100000:2.0:1789762976.414714:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499718466049 time 499718588048 state 0x0 ref 1 00000100:00100000:2.0:1789762976.414752:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f2b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015187840:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762976.414806:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b4ae57d/0x416193e012b0632e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762976.414832:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762976.414839:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762976.414850:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b4ae57d/0x416193e012b0632e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762976.414873:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b4ae57d/0x416193e012b0632e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.418350:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b4ae57d/0x416193e012b0632e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.418365:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b4ae57d/0x416193e012b0632e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.418377:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b4ae57d/0x416193e012b0632e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.418386:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83426e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762976.418391:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b4ae57d/0x416193e012b0632e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.418400:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b4ae57d/0x416193e012b0632e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.418422:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b4ae57d/0x416193e012b0632e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.418430:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b4ae57d/0x416193e012b0632e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [77594624->78643199] (req 77594624->78643199) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762976.418499:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f2b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015187840:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3762us (96648us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762976.418509:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1562 00000100:00100000:3.0:1789762976.418522:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499718466049 time 499718588048 state 0x2 ref 0 00000100:00100000:0.0:1789762976.420882:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762976.420931:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015187968 00000100:00100000:3.0:1789762976.420943:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015187968 00000100:00100000:3.0:1789762976.420994:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499718466049 time 499718588048 state 0x0 ref 1 00000100:00100000:3.0:1789762976.421013:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499818588048 time 499718588048 state 0x1 ref 1 00000100:00100000:1.0:1789762976.514840:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499818588048 time 499818726998 state 0x0 ref 1 00000100:00100000:1.0:1789762976.514876:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015187968:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762976.514959:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b06335 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.514982:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762976.514988:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762976.514995:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b06335 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.515012:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b06335 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.518986:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b06335 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.519015:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b06335 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.519029:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b06335 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.519040:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9f774400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762976.519048:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b06335 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.519126:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b06335 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.519152:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b06335 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.519160:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b06335 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [78643200->79691775] (req 78643200->79691775) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762976.519234:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015187968:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4370us (98351us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762976.519244:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1563 00000100:00100000:1.0:1789762976.519256:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499818588048 time 499818726998 state 0x2 ref 0 00000100:00100000:1.0:1789762976.521649:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762976.521715:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015188096 00000100:00100000:3.0:1789762976.521743:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015188096 00000100:00100000:3.0:1789762976.521796:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499818588048 time 499818726998 state 0x0 ref 1 00000100:00100000:3.0:1789762976.521812:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499918726998 time 499818726998 state 0x1 ref 1 00000100:00100000:1.0:1789762976.614976:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499918726998 time 499918854626 state 0x0 ref 1 00000100:00100000:1.0:1789762976.615037:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015188096:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762976.615166:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b0633c lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.615187:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762976.615191:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762976.615198:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b0633c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.615213:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b0633c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762976.618680:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b0633c lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762976.618692:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b0633c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762976.618701:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b0633c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762976.618708:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9f774400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762976.618712:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b0633c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762976.618721:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b0633c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762976.618739:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b0633c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762976.618745:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003f2d2015/0x416193e012b0633c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [79691776->80740351] (req 79691776->80740351) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762976.618811:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015188096:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3809us (97164us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762976.618823:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1564 00000100:00100000:0.0:1789762976.618836:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499918726998 time 499918854626 state 0x2 ref 0 00000100:00100000:1.0:1789762976.621512:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762976.621630:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015188224 00000100:00100000:3.0:1789762976.621654:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015188224 00000100:00100000:3.0:1789762976.621709:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 499918726998 time 499918854626 state 0x0 ref 1 00000100:00100000:3.0:1789762976.621722:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500018854626 time 499918854626 state 0x1 ref 1 00010000:00010000:0.0:1789762976.695270:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762976.715215:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500018854626 time 500019089196 state 0x0 ref 1 00000100:00100000:1.0:1789762976.715250:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015188224:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762976.715326:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000651ed65f/0x416193e012b06343 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.715349:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762976.715355:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762976.715363:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000651ed65f/0x416193e012b06343 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762976.715412:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000651ed65f/0x416193e012b06343 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.718802:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000651ed65f/0x416193e012b06343 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.718819:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000651ed65f/0x416193e012b06343 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.718826:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000651ed65f/0x416193e012b06343 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.718831:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e9f775a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762976.718835:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000651ed65f/0x416193e012b06343 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.718842:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000651ed65f/0x416193e012b06343 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.718857:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000651ed65f/0x416193e012b06343 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.718862:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000651ed65f/0x416193e012b06343 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [80740352->81788927] (req 80740352->81788927) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762976.718935:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015188224:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3697us (97442us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762976.718944:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1565 00000100:00100000:3.0:1789762976.718954:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500018854626 time 500019089196 state 0x2 ref 0 00000100:00100000:2.0:1789762976.721075:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762976.721141:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015188352 00000100:00100000:3.0:1789762976.721150:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015188352 00000100:00100000:3.0:1789762976.721190:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500018854626 time 500019089196 state 0x0 ref 1 00000100:00100000:3.0:1789762976.721201:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500119089196 time 500019089196 state 0x1 ref 1 00000100:00100000:3.0:1789762976.815253:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500119089196 time 500119149593 state 0x0 ref 1 00000100:00100000:3.0:1789762976.815277:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015188352:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762976.815316:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0634a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.815341:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762976.815345:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762976.815350:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0634a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.815360:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0634a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.818638:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0634a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.818649:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0634a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.818656:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0634a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.818662:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762976.818666:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0634a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.818672:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0634a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.818690:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0634a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.818695:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0634a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [81788928->82837503] (req 81788928->82837503) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762976.818742:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f28e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015188352:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3473us (97667us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762976.818748:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1566 00000100:00100000:3.0:1789762976.818755:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500119089196 time 500119149593 state 0x2 ref 0 00000100:00100000:0.0:1789762976.821149:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762976.821185:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015188480 00000100:00100000:3.0:1789762976.821193:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015188480 00000100:00100000:3.0:1789762976.821227:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500119089196 time 500119149593 state 0x0 ref 1 00000100:00100000:3.0:1789762976.821238:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500219149593 time 500119149593 state 0x1 ref 1 00000100:00100000:2.0:1789762976.866264:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762976.866296:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762976.866320:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762976.866321:0:5797:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015188608 00000100:00100000:0.0:1789762976.866331:0:5797:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015188608 00000100:00100000:2.0:1789762976.866356:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762976.866357:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015188736 00000100:00100000:1.0:1789762976.866363:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015188736 00000100:00100000:0.0:1789762976.866383:0:5797:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 368 00000100:00100000:3.0:1789762976.866386:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015188992 00000100:00100000:3.0:1789762976.866395:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015188992 00000100:00100000:0.0:1789762976.866397:0:5797:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85084850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015188608:12345-192.168.203.48@tcp:400:kworker.0 00000100:00080000:2.0:1789762976.866404:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762976.866419:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 407 00000100:00100000:1.0:1789762976.866429:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f28a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2366:x1876702015188736:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:2.0:1789762976.866437:0:6881:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015188864 00000100:00100000:2.0:1789762976.866443:0:6881:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015188864 00000100:00100000:3.0:1789762976.866450:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 408 00000100:00100000:0.0:1789762976.866463:0:5797:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85084850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:9e30d9c2-532c-4e55-80be-13e95f7c85ac+13:2365:x1876702015188608:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 68us (203us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762976.866464:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015188992:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:1.0:1789762976.866472:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f28a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:2366:x1876702015188736:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 41us (170us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762976.866479:0:5797:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 368 00000100:00100000:1.0:1789762976.866483:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 407 00000100:00100000:2.0:1789762976.866498:0:6881:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 201 00000100:00100000:3.0:1789762976.866511:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f28000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:871a7a42-3659-47e5-8fc9-f4f1e262590d+5:2366:x1876702015188992:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 50us (186us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762976.866518:0:6881:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015188864:12345-192.168.203.48@tcp:400:kworker.0 00000100:00100000:3.0:1789762976.866532:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 408 00000100:00100000:2.0:1789762976.866557:0:6881:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f29500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:2365:x1876702015188864:12345-192.168.203.48@tcp:400:kworker.0 Request processed in 51us (201us total) trans 0 rc 0/0 00000100:00080000:3.0:1789762976.866569:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789762976.866581:0:6881:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 201 00000100:00100000:3.0:1789762976.887268:0:3274:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789762976.887287:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ea0f2a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022204032:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762976.887341:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762976.887390:0:8994:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022204032 00000100:00100000:0.0:1789762976.887399:0:8994:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022204032 00000100:00080000:3.0:1789762976.887446:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789762976.887465:0:8994:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:0.0:1789762976.887474:0:8994:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8342ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022204032:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789762976.887513:0:8994:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 526581760 pending 0 free 3754950656 avail 3752853504 00000100:00100000:0.0:1789762976.887560:0:8994:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8342ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3274:x1876702022204032:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (217us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762976.887570:0:8994:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:3.0:1789762976.887586:0:3274:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294968369 gen 2 00000004:00080000:3.0:1789762976.887592:0:3274:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981e8342ea00, transno 4294968369 00000100:00100000:3.0:1789762976.887617:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ea0f2a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3274:1876702022204032:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789762976.915312:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500219149593 time 500219210067 state 0x0 ref 1 00000100:00100000:3.0:1789762976.915331:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015188480:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762976.915363:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06351 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.915377:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762976.915381:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762976.915386:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06351 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.915397:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06351 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.918775:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06351 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.918787:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06351 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.918795:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06351 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.918801:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762976.918804:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06351 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.918810:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06351 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.918824:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06351 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762976.918829:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06351 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [82837504->83886079] (req 82837504->83886079) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762976.918879:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015188480:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3552us (97732us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762976.918885:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1567 00000100:00100000:3.0:1789762976.918893:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500219149593 time 500219210067 state 0x2 ref 0 00000100:00100000:2.0:1789762976.920975:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762976.921034:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015189120 00000100:00100000:1.0:1789762976.921044:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015189120 00000100:00100000:1.0:1789762976.921116:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500219149593 time 500219210067 state 0x0 ref 1 00000100:00100000:1.0:1789762976.921128:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500319210067 time 500219210067 state 0x1 ref 1 00000100:00100000:3.0:1789762977.015409:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500319210067 time 500319300157 state 0x0 ref 1 00000100:00100000:3.0:1789762977.015457:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015189120:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762977.015509:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06358 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789762977.015514:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789762977.015529:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.015534:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.015542:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06358 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.015557:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06358 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.018689:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06358 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.018704:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06358 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.018716:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06358 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.018725:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762977.018731:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06358 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.018743:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06358 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.018768:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06358 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.018778:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06358 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [83886080->84934655] (req 83886080->84934655) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762977.018845:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f2b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015189120:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3416us (97870us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762977.018856:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1568 00000100:00100000:3.0:1789762977.018867:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500319210067 time 500319300157 state 0x2 ref 0 00000100:00100000:0.0:1789762977.020756:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762977.020803:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015189248 00000100:00100000:1.0:1789762977.020812:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015189248 00000100:00100000:1.0:1789762977.020847:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500319210067 time 500319300157 state 0x0 ref 1 00000100:00100000:1.0:1789762977.020859:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500419300157 time 500319300157 state 0x1 ref 1 00000100:00100000:3.0:1789762977.115490:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500419300157 time 500419385750 state 0x0 ref 1 00000100:00100000:3.0:1789762977.115511:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015189248:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762977.115545:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0635f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.115559:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.115574:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.115581:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0635f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.115596:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0635f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.119206:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0635f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.119218:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0635f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.119224:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0635f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.119229:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762977.119233:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0635f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.119239:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0635f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.119252:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0635f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.119257:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b0635f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [84934656->85983231] (req 84934656->85983231) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762977.119304:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015189248:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3799us (98548us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762977.119311:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1569 00000100:00100000:3.0:1789762977.119320:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500419300157 time 500419385750 state 0x2 ref 0 00000100:00100000:1.0:1789762977.121428:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762977.121484:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015189376 00000100:00100000:2.0:1789762977.121494:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015189376 00000100:00100000:2.0:1789762977.121534:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500419300157 time 500419385750 state 0x0 ref 1 00000100:00100000:2.0:1789762977.121545:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500519385750 time 500419385750 state 0x1 ref 1 00000100:00100000:3.0:1789762977.215588:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500519385750 time 500519478368 state 0x0 ref 1 00000100:00100000:3.0:1789762977.215620:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015189376:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762977.215659:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06366 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.215682:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.215685:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.215690:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06366 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.215702:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06366 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.218836:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06366 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.218845:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06366 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.218852:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06366 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.218858:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762977.218874:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06366 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.218881:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06366 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.218896:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06366 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.218901:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06366 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [85983232->87031807] (req 85983232->87031807) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762977.218946:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015189376:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3340us (97518us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762977.218952:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1570 00000100:00100000:3.0:1789762977.218959:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500519385750 time 500519478368 state 0x2 ref 0 00000100:00100000:2.0:1789762977.221231:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762977.221286:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015189504 00000100:00100000:1.0:1789762977.221296:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015189504 00000100:00100000:1.0:1789762977.221341:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500519385750 time 500519478368 state 0x0 ref 1 00000100:00100000:1.0:1789762977.221356:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500619478368 time 500519478368 state 0x1 ref 1 00000100:00100000:1.0:1789762977.315666:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500619478368 time 500619557047 state 0x0 ref 1 00000100:00100000:1.0:1789762977.315696:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f28700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015189504:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762977.315739:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b0636d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.315757:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762977.315761:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762977.315767:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b0636d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.315782:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b0636d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.319125:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b0636d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.319138:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b0636d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.319148:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b0636d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.319168:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762977.319173:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b0636d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.319182:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b0636d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.319201:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b0636d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.319210:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b0636d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [87031808->88080383] (req 87031808->88080383) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762977.319268:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f28700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015189504:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3583us (98038us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762977.319279:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1571 00000100:00100000:1.0:1789762977.319291:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500619478368 time 500619557047 state 0x2 ref 0 00000100:00100000:0.0:1789762977.321513:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762977.321557:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015189632 00000100:00100000:1.0:1789762977.321566:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015189632 00000100:00100000:1.0:1789762977.321605:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500619478368 time 500619557047 state 0x0 ref 1 00000100:00100000:1.0:1789762977.321620:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500719557047 time 500619557047 state 0x1 ref 1 00000100:00100000:2.0:1789762977.399271:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022204160:0@lo:400:kworker.0 00000100:00100000:0.0:1789762977.399273:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981e8342c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022204288:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762977.399277:0:3272:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb32a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022204416:0@lo:400:kworker.0 00000100:00100000:2.0:1789762977.399362:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762977.399364:0:3272:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762977.399368:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762977.399409:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022204416 00000100:00100000:3.0:1789762977.399417:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022204416 00000100:00100000:2.0:1789762977.399428:0:3274:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb31180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022204672:0@lo:400:kworker.0 00000100:00100000:0.0:1789762977.399430:0:3271:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb33100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022204800:0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.399449:0:3273:0:(client.c:1919:ptlrpc_send_new_req()) Sending RPC req@ffff981ebfb32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022204544:0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.399464:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1789762977.399472:0:3274:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789762977.399478:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f7d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022204416:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789762977.399492:0:3271:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789762977.399503:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022204672 00000100:00100000:1.0:1789762977.399510:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022204672 00000100:00100000:3.0:1789762977.399564:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f7d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3272:x1876702022204416:12345-0@lo:400:kworker.0 Request processed in 91us (206us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762977.399567:0:10916:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022204800 00000100:00080000:2.0:1789762977.399570:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789762977.399573:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:0.0:1789762977.399574:0:10916:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022204800 00000100:00100000:1.0:1789762977.399588:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:1.0:1789762977.399611:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022204672:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.399626:0:5799:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022204160 00000100:00100000:3.0:1789762977.399632:0:5799:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022204160 00000100:00100000:2.0:1789762977.399633:0:3273:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789762977.399647:0:10916:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:0.0:1789762977.399658:0:10916:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022204800:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.399673:0:5799:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:1.0:1789762977.399678:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f29c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3274:x1876702022204672:12345-0@lo:400:kworker.0 Request processed in 78us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762977.399688:0:5799:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e85086450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022204160:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.399702:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:2.0:1789762977.399716:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb31180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3274:1876702022204672:0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.399726:0:5806:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022204544 00000100:00100000:0.0:1789762977.399733:0:10916:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022204800:12345-0@lo:400:kworker.0 Request processed in 75us (239us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762977.399734:0:5806:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022204544 00000100:00100000:3.0:1789762977.399744:0:5799:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e85086450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:bbe4e1bd-8437-4448-83b9-0b3f7551a439+19:3274:x1876702022204160:12345-0@lo:400:kworker.0 Request processed in 58us (386us total) trans 0 rc 0/0 00000100:00100000:0.0:1789762977.399751:0:10916:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:2.0:1789762977.399751:0:3274:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:bbe4e1bd-8437-4448-83b9-0b3f7551a439:3274:1876702022204160:0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.399762:0:5799:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00000100:00100000:1.0:1789762977.399774:0:5806:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00080000:0.0:1789762977.399785:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789762977.399789:0:5806:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea0f2a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022204544:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789762977.399792:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789762977.399805:0:6651:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702022204288 00000100:00100000:3.0:1789762977.399811:0:6651:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702022204288 00000100:00100000:0.0:1789762977.399825:0:3271:0:(client.c:3125:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967452 gen 2 00000004:00080000:0.0:1789762977.399832:0:3271:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff981e83423480, transno 4294967452 00000100:00100000:2.0:1789762977.399838:0:3272:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb32a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3272:1876702022204416:0@lo:400:kworker.0 00000100:00100000:0.0:1789762977.399852:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb33100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022204800:0@lo:400:kworker.0 00000100:00100000:3.0:1789762977.399853:0:6651:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:3.0:1789762977.399869:0:6651:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022204288:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762977.399871:0:5806:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea0f2a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3273:x1876702022204544:12345-0@lo:400:kworker.0 Request processed in 85us (240us total) trans 0 rc 0/0 00000100:00100000:2.0:1789762977.399879:0:3273:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981ebfb32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3273:1876702022204544:0@lo:400:kworker.0 00000100:00100000:1.0:1789762977.399895:0:5806:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00002000:00000024:3.0:1789762977.399925:0:6651:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 6815744 granted 71499776 pending 0 free 3859808256 avail 3857711104 00000100:00100000:3.0:1789762977.399970:0:6651:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3271:x1876702022204288:12345-0@lo:13:osp-pre-1-0.0 Request processed in 105us (612us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762977.399983:0:6651:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00100000:1.0:1789762977.400010:0:3271:0:(client.c:2414:ptlrpc_check_set()) Completed RPC req@ffff981e8342c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3271:1876702022204288:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789762977.415727:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500719557047 time 500719623497 state 0x0 ref 1 00000100:00100000:1.0:1789762977.415746:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015189632:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762977.415778:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b06374 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.415795:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762977.415799:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762977.415807:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b06374 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.415822:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b06374 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.418935:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b06374 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.418947:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b06374 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.418958:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b06374 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.418967:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e880ab200 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762977.418972:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b06374 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.418982:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b06374 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.419001:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b06374 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.419011:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000058de5a3/0x416193e012b06374 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [88080384->89128959] (req 88080384->89128959) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762977.419082:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015189632:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3339us (97579us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762977.419125:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1572 00000100:00100000:1.0:1789762977.419136:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500719557047 time 500719623497 state 0x2 ref 0 00000100:00100000:0.0:1789762977.421207:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762977.421250:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015189760 00000100:00100000:3.0:1789762977.421257:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015189760 00000100:00100000:3.0:1789762977.421291:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500719557047 time 500719623497 state 0x0 ref 1 00000100:00100000:3.0:1789762977.421303:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500819623497 time 500719623497 state 0x1 ref 1 00000100:00100000:2.0:1789762977.515841:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500819623497 time 500819725680 state 0x0 ref 1 00000100:00100000:2.0:1789762977.515866:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015189760:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762977.515925:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0637b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.515939:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762977.515943:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762977.515948:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0637b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.515967:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0637b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.519275:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0637b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.519291:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0637b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.519301:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0637b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.519315:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f09800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762977.519319:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0637b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.519326:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0637b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.519340:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0637b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.519346:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d82fa3ab/0x416193e012b0637b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [89128960->90177535] (req 89128960->90177535) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762977.519394:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015189760:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3538us (98189us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762977.519401:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1573 00000100:00100000:3.0:1789762977.519409:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500819623497 time 500819725680 state 0x2 ref 0 00000100:00100000:0.0:1789762977.521510:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762977.521545:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015189888 00000100:00100000:3.0:1789762977.521554:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015189888 00000100:00100000:3.0:1789762977.521587:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500819623497 time 500819725680 state 0x0 ref 1 00000100:00100000:3.0:1789762977.521598:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500919725680 time 500819725680 state 0x1 ref 1 00000100:00100000:1.0:1789762977.615952:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500919725680 time 500919843380 state 0x0 ref 1 00000100:00100000:1.0:1789762977.615983:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015189888:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762977.616023:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005feaac96/0x416193e012b06382 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.616040:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762977.616061:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762977.616069:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005feaac96/0x416193e012b06382 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.616084:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005feaac96/0x416193e012b06382 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.619396:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000005feaac96/0x416193e012b06382 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.619409:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000005feaac96/0x416193e012b06382 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.619417:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000005feaac96/0x416193e012b06382 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.619423:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83425000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762977.619427:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000005feaac96/0x416193e012b06382 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.619433:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000005feaac96/0x416193e012b06382 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.619459:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000005feaac96/0x416193e012b06382 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762977.619465:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005feaac96/0x416193e012b06382 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [90177536->91226111] (req 90177536->91226111) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762977.619525:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015189888:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3554us (98015us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762977.619532:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1574 00000100:00100000:2.0:1789762977.619552:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500919725680 time 500919843380 state 0x2 ref 0 00000100:00100000:0.0:1789762977.621544:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762977.621585:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015190016 00000100:00100000:3.0:1789762977.621593:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015190016 00000100:00100000:3.0:1789762977.621629:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 500919725680 time 500919843380 state 0x0 ref 1 00000100:00100000:3.0:1789762977.621640:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501019843380 time 500919843380 state 0x1 ref 1 00000100:00100000:1.0:1789762977.716063:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501019843380 time 501019947162 state 0x0 ref 1 00000100:00100000:1.0:1789762977.716118:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015190016:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762977.716155:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b06389 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.716170:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789762977.716174:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762977.716179:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b06389 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.716191:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b06389 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.719238:0:8466:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762977.719344:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b06389 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.719367:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b06389 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.719375:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b06389 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.719382:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83424a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762977.719386:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b06389 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.719395:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b06389 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.719412:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b06389 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.719420:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b06389 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [91226112->92274687] (req 91226112->92274687) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762977.719485:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015190016:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3403us (97939us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762977.719495:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1575 00000100:00100000:1.0:1789762977.719507:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501019843380 time 501019947162 state 0x2 ref 0 00000100:00100000:3.0:1789762977.721761:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762977.721807:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015190144 00000100:00100000:2.0:1789762977.721815:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015190144 00000100:00100000:2.0:1789762977.721854:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501019843380 time 501019947162 state 0x0 ref 1 00000100:00100000:2.0:1789762977.721865:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501119947162 time 501019947162 state 0x1 ref 1 00000100:00100000:3.0:1789762977.816169:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501119947162 time 501120064563 state 0x0 ref 1 00000100:00100000:3.0:1789762977.816197:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8342d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015190144:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762977.816236:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06390 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.816249:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.816253:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.816257:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06390 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.816269:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06390 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.819635:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06390 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.819646:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06390 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.819653:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06390 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.819658:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f19400 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762977.819662:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06390 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.819668:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06390 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.819681:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06390 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762977.819688:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c2173b13/0x416193e012b06390 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [92274688->93323263] (req 92274688->93323263) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762977.819753:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8342d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015190144:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3566us (97991us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762977.819762:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1576 00000100:00100000:1.0:1789762977.819770:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501119947162 time 501120064563 state 0x2 ref 0 00000100:00100000:2.0:1789762977.821916:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762977.821955:0:17497:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015190272 00000100:00100000:1.0:1789762977.821964:0:17497:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015190272 00000100:00100000:1.0:1789762977.822001:0:17497:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501119947162 time 501120064563 state 0x0 ref 1 00000100:00100000:1.0:1789762977.822012:0:17497:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501220064563 time 501120064563 state 0x1 ref 1 00000100:00100000:3.0:1789762977.916240:0:6654:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501220064563 time 501220127238 state 0x0 ref 1 00000100:00100000:3.0:1789762977.916261:0:6654:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e83423480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015190272:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762977.916294:0:6654:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b06397 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.916313:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.916317:0:6654:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762977.916321:0:6654:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b06397 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.916332:0:6654:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b06397 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.919287:0:6654:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b06397 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.919297:0:6654:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b06397 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.919304:0:6654:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b06397 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.919309:0:6654:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f18000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762977.919312:0:6654:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b06397 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.919318:0:6654:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b06397 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.919331:0:6654:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b06397 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762977.919340:0:6654:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b06397 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [93323264->94371839] (req 93323264->94371839) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6654 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762977.919384:0:6654:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e83423480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015190272:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3129us (97469us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762977.919391:0:6654:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1577 00000100:00100000:3.0:1789762977.919398:0:6654:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501220064563 time 501220127238 state 0x2 ref 0 00000100:00100000:2.0:1789762977.921480:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762977.921559:0:17498:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015190400 00000100:00100000:0.0:1789762977.921568:0:17498:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015190400 00000100:00100000:0.0:1789762977.921603:0:17498:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501220064563 time 501220127238 state 0x0 ref 1 00000100:00100000:0.0:1789762977.921614:0:17498:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501320127238 time 501220127238 state 0x1 ref 1 00000100:00100000:3.0:1789762978.016314:0:6653:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501320127238 time 501320210058 state 0x0 ref 1 00000100:00100000:3.0:1789762978.016340:0:6653:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e84a41500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015190400:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762978.016376:0:6653:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b0639e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.016391:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762978.016394:0:6653:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762978.016399:0:6653:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b0639e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.016410:0:6653:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b0639e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.019430:0:6653:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b0639e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.019446:0:6653:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b0639e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.019454:0:6653:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b0639e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.019459:0:6653:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981ea0f18000 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762978.019463:0:6653:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b0639e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.019470:0:6653:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b0639e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.019484:0:6653:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b0639e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.019489:0:6653:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000415a5728/0x416193e012b0639e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [94371840->95420415] (req 94371840->95420415) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6653 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762978.019539:0:6653:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e84a41500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015190400:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3210us (98059us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762978.019546:0:6653:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1578 00000100:00100000:3.0:1789762978.019553:0:6653:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501320127238 time 501320210058 state 0x2 ref 0 00000100:00100000:2.0:1789762978.021351:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762978.021400:0:14913:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015190528 00000100:00100000:1.0:1789762978.021408:0:14913:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015190528 00000100:00100000:1.0:1789762978.021443:0:14913:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501320127238 time 501320210058 state 0x0 ref 1 00000100:00100000:1.0:1789762978.021454:0:14913:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501420210058 time 501320210058 state 0x1 ref 1 00000100:00100000:2.0:1789762978.116385:0:14914:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501420210058 time 501420280672 state 0x0 ref 1 00000100:00100000:2.0:1789762978.116406:0:14914:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e84a42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015190528:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762978.116444:0:14914:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b063a5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.116475:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762978.116480:0:14914:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762978.116488:0:14914:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b063a5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.116503:0:14914:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b063a5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.120650:0:14914:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b063a5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.120666:0:14914:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b063a5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.120672:0:14914:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b063a5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.120678:0:14914:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81482e00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:2.0:1789762978.120681:0:14914:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b063a5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.120688:0:14914:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b063a5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.120703:0:14914:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b063a5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.120720:0:14914:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ea78e1d7/0x416193e012b063a5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [95420416->96468991] (req 95420416->96468991) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14914 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789762978.120779:0:14914:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e84a42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_008:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015190528:12345-192.168.203.48@tcp:3:dd.500 Request processed in 4378us (99426us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789762978.120787:0:14914:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1579 00000100:00100000:2.0:1789762978.120799:0:14914:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501420210058 time 501420280672 state 0x2 ref 0 00000100:00100000:2.0:1789762978.122996:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762978.123057:0:11723:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015190656 00000100:00100000:1.0:1789762978.123065:0:11723:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015190656 00000100:00100000:1.0:1789762978.123132:0:11723:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501420210058 time 501420280672 state 0x0 ref 1 00000100:00100000:1.0:1789762978.123144:0:11723:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501520280672 time 501420280672 state 0x1 ref 1 00000100:00100000:1.0:1789762978.216495:0:17497:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501520280672 time 501520369177 state 0x0 ref 1 00000100:00100000:1.0:1789762978.216530:0:17497:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e84a43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015190656:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762978.216572:0:17497:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ac lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.216590:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762978.216595:0:17497:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762978.216601:0:17497:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ac lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.216615:0:17497:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ac lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.220289:0:17497:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ac lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.220305:0:17497:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ac lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.220312:0:17497:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ac lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.220320:0:17497:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83424a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762978.220326:0:17497:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ac lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.220337:0:17497:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ac lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.220359:0:17497:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ac lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.220368:0:17497:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ac lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [96468992->97517567] (req 96468992->97517567) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17497 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762978.220426:0:17497:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e84a43480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_009:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015190656:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3911us (97430us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762978.220436:0:17497:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1580 00000100:00100000:1.0:1789762978.220445:0:17497:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501520280672 time 501520369177 state 0x2 ref 0 00000100:00100000:2.0:1789762978.222416:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762978.222466:0:12582:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015190784 00000100:00100000:3.0:1789762978.222474:0:12582:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015190784 00000100:00100000:3.0:1789762978.222511:0:12582:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501520280672 time 501520369177 state 0x0 ref 1 00000100:00100000:3.0:1789762978.222530:0:12582:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501620369177 time 501520369177 state 0x1 ref 1 00000100:00100000:0.0:1789762978.316558:0:17498:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501620369177 time 501620451397 state 0x0 ref 1 00000100:00100000:0.0:1789762978.316589:0:17498:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e84a40700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015190784:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:0.0:1789762978.316630:0:17498:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d7b7cfd2/0x416193e012b063b3 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762978.316643:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762978.316647:0:17498:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789762978.316651:0:17498:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d7b7cfd2/0x416193e012b063b3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762978.316663:0:17498:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d7b7cfd2/0x416193e012b063b3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762978.319695:0:17498:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d7b7cfd2/0x416193e012b063b3 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762978.319704:0:17498:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d7b7cfd2/0x416193e012b063b3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762978.319714:0:17498:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d7b7cfd2/0x416193e012b063b3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762978.319719:0:17498:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87ca1c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:0.0:1789762978.319723:0:17498:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7b7cfd2/0x416193e012b063b3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762978.319729:0:17498:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d7b7cfd2/0x416193e012b063b3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762978.319742:0:17498:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d7b7cfd2/0x416193e012b063b3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789762978.319747:0:17498:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d7b7cfd2/0x416193e012b063b3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [97517568->98566143] (req 97517568->98566143) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 17498 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762978.319792:0:17498:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e84a40700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_010:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015190784:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3216us (97376us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789762978.319799:0:17498:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1581 00000100:00100000:0.0:1789762978.319807:0:17498:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501620369177 time 501620451397 state 0x2 ref 0 00000100:00100000:0.0:1789762978.321640:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762978.321720:0:14912:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015190912 00000100:00100000:2.0:1789762978.321729:0:14912:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015190912 00000100:00100000:2.0:1789762978.321766:0:14912:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501620369177 time 501620451397 state 0x0 ref 1 00000100:00100000:2.0:1789762978.321776:0:14912:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501720451397 time 501620451397 state 0x1 ref 1 00000100:00100000:1.0:1789762978.416643:0:14913:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501720451397 time 501720538418 state 0x0 ref 1 00000100:00100000:1.0:1789762978.416663:0:14913:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015190912:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762978.416713:0:14913:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ba lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.416731:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762978.416737:0:14913:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762978.416744:0:14913:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ba lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.416772:0:14913:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ba lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.419857:0:14913:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ba lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.419867:0:14913:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ba lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.419874:0:14913:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ba lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.419880:0:14913:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83424a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762978.419884:0:14913:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ba lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.419891:0:14913:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ba lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.419905:0:14913:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ba lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.419911:0:14913:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063ba lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [98566144->99614719] (req 98566144->99614719) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14913 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762978.419961:0:14913:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_007:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015190912:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3303us (98321us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762978.419968:0:14913:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1582 00000100:00100000:1.0:1789762978.419975:0:14913:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501720451397 time 501720538418 state 0x2 ref 0 00000100:00100000:0.0:1789762978.421904:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762978.421942:0:6652:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015191040 00000100:00100000:1.0:1789762978.421949:0:6652:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015191040 00000100:00100000:1.0:1789762978.421984:0:6652:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501720451397 time 501720538418 state 0x0 ref 1 00000100:00100000:1.0:1789762978.421995:0:6652:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501820538418 time 501720538418 state 0x1 ref 1 00000100:00100000:1.0:1789762978.516700:0:11723:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501820538418 time 501820596340 state 0x0 ref 1 00000100:00100000:1.0:1789762978.516723:0:11723:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015191040:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762978.516759:0:11723:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063c1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.516773:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762978.516777:0:11723:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762978.516781:0:11723:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063c1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.516792:0:11723:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063c1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.519878:0:11723:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063c1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.519889:0:11723:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063c1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.519895:0:11723:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063c1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.519901:0:11723:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83424a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762978.519904:0:11723:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063c1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.519911:0:11723:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063c1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.519923:0:11723:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063c1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.519928:0:11723:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063c1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [99614720->100663295] (req 99614720->100663295) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 11723 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762978.519976:0:11723:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015191040:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3261us (98072us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762978.519984:0:11723:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1583 00000100:00100000:1.0:1789762978.519993:0:11723:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501820538418 time 501820596340 state 0x2 ref 0 00000100:00100000:0.0:1789762978.521855:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762978.521888:0:12581:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015191168 00000100:00100000:3.0:1789762978.521896:0:12581:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015191168 00000100:00100000:3.0:1789762978.521931:0:12581:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501820538418 time 501820596340 state 0x0 ref 1 00000100:00100000:3.0:1789762978.521942:0:12581:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501920596340 time 501820596340 state 0x1 ref 1 00000100:00100000:3.0:1789762978.616802:0:12582:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501920596340 time 501920697137 state 0x0 ref 1 00000100:00100000:3.0:1789762978.616828:0:12582:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015191168:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762978.616865:0:12582:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b063c8 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.616879:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762978.616882:0:12582:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762978.616887:0:12582:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b063c8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.616897:0:12582:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b063c8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.620231:0:12582:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b063c8 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.620243:0:12582:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b063c8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.620251:0:12582:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b063c8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.620257:0:12582:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e81483600 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762978.620262:0:12582:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b063c8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.620268:0:12582:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b063c8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.620283:0:12582:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b063c8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.620288:0:12582:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000011240690/0x416193e012b063c8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [100663296->101711871] (req 100663296->101711871) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12582 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762978.620335:0:12582:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_005:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015191168:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3518us (98480us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762978.620342:0:12582:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1584 00000100:00100000:3.0:1789762978.620350:0:12582:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501920596340 time 501920697137 state 0x2 ref 0 00000100:00100000:3.0:1789762978.622510:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:0.0:1789762978.622552:0:6654:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015191296 00000100:00100000:0.0:1789762978.622563:0:6654:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015191296 00000100:00100000:0.0:1789762978.622606:0:6654:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 501920596340 time 501920697137 state 0x0 ref 1 00000100:00100000:0.0:1789762978.622625:0:6654:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 502020697137 time 501920697137 state 0x1 ref 1 00000100:00100000:2.0:1789762978.716881:0:14912:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 502020697137 time 502020778352 state 0x0 ref 1 00000100:00100000:2.0:1789762978.716904:0:14912:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8342e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015191296:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:2.0:1789762978.716941:0:14912:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b7302be2/0x416193e012b063cf lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.716955:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789762978.716958:0:14912:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789762978.716976:0:14912:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b7302be2/0x416193e012b063cf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.716994:0:14912:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b7302be2/0x416193e012b063cf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.720110:0:14912:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b7302be2/0x416193e012b063cf lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.720122:0:14912:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b7302be2/0x416193e012b063cf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.720128:0:14912:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b7302be2/0x416193e012b063cf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.720133:0:14912:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83451800 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762978.720137:0:14912:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7302be2/0x416193e012b063cf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.720143:0:14912:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b7302be2/0x416193e012b063cf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.720156:0:14912:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b7302be2/0x416193e012b063cf lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.720161:0:14912:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b7302be2/0x416193e012b063cf lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [101711872->102760447] (req 101711872->102760447) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 14912 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762978.720211:0:14912:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8342e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_006:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015191296:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3313us (97700us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762978.720218:0:14912:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1585 00000100:00100000:1.0:1789762978.720225:0:14912:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 502020697137 time 502020778352 state 0x2 ref 0 00000100:00100000:2.0:1789762978.722391:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762978.722446:0:6653:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015191424 00000100:00100000:3.0:1789762978.722456:0:6653:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015191424 00000100:00100000:3.0:1789762978.722500:0:6653:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 502020697137 time 502020778352 state 0x0 ref 1 00000100:00100000:3.0:1789762978.722511:0:6653:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 502120778352 time 502020778352 state 0x1 ref 1 00010000:00010000:2.0:1789762978.743202:0:6663:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789762978.816966:0:6652:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 502120778352 time 502120861525 state 0x0 ref 1 00000100:00100000:1.0:1789762978.816998:0:6652:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e84a41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015191424:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:1.0:1789762978.817046:0:6652:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063d6 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.817060:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762978.817064:0:6652:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789762978.817069:0:6652:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063d6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.817115:0:6652:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063d6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.820136:0:6652:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063d6 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.820154:0:6652:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063d6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.820161:0:6652:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063d6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.820167:0:6652:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83424a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762978.820171:0:6652:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063d6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.820178:0:6652:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063d6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.820193:0:6652:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063d6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.820199:0:6652:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000098e62d6c/0x416193e012b063d6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [102760448->103809023] (req 102760448->103809023) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6652 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762978.820246:0:6652:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e84a41c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2364:x1876702015191424:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3257us (97857us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789762978.820254:0:6652:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1586 00000100:00100000:1.0:1789762978.820261:0:6652:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 502120778352 time 502120861525 state 0x2 ref 0 00000100:00100000:0.0:1789762978.822196:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762978.822238:0:14914:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015191552 00000100:00100000:2.0:1789762978.822248:0:14914:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015191552 00000100:00100000:2.0:1789762978.822293:0:14914:0:(nrs_tbf.c:1485:nrs_tbf_lru_tryhit()) ost_io.0 NRS: TBF LRU hit: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 502120778352 time 502120861525 state 0x0 ref 1 00000100:00100000:2.0:1789762978.822305:0:14914:0:(nrs_tbf.c:2897:nrs_tbf_req_add()) ost_io.0 NRS: TBF enqueues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 502220861525 time 502120861525 state 0x1 ref 1 00000100:00100000:3.0:1789762978.917081:0:12581:0:(nrs_tbf.c:2821:nrs_tbf_req_get()) ost_io.0 NRS: TBF dequeues: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 502220861525 time 502220971667 state 0x0 ref 1 00000100:00100000:3.0:1789762978.917125:0:12581:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ebfb31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015191552:12345-192.168.203.48@tcp:3:dd.500 00010000:00010000:3.0:1789762978.917160:0:12581:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000323e39c3/0x416193e012b063dd lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.917175:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762978.917178:0:12581:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789762978.917183:0:12581:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000323e39c3/0x416193e012b063dd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.917194:0:12581:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000323e39c3/0x416193e012b063dd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.920071:0:12581:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000323e39c3/0x416193e012b063dd lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.920084:0:12581:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000323e39c3/0x416193e012b063dd lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.920122:0:12581:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000323e39c3/0x416193e012b063dd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.920132:0:12581:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87ca0a00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:3.0:1789762978.920137:0:12581:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000323e39c3/0x416193e012b063dd lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.920147:0:12581:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000323e39c3/0x416193e012b063dd lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.920163:0:12581:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000323e39c3/0x416193e012b063dd lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.920169:0:12581:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000323e39c3/0x416193e012b063dd lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [103809024->104857599] (req 103809024->104857599) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 12581 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762978.920215:0:12581:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ebfb31180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:2363:x1876702015191552:12345-192.168.203.48@tcp:3:dd.500 Request processed in 3100us (98020us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789762978.920221:0:12581:0:(nrs_tbf.c:2952:nrs_tbf_req_stop()) NRS stop tbf request from 12345-192.168.203.48@tcp, seq: 1587 00000100:00100000:3.0:1789762978.920228:0:12581:0:(nrs_tbf.c:1465:nrs_tbf_lru_add()) ost_io.0 NRS: TBF LRU add: class@ffff981d8392ea00 key jobid={dd.500}&nid={192.168.203.48@tcp}&opcode={ost_read}&uid={500}&gid={500}&projid={0}&nodemap={-1} rule@0xffff981d8a978a00 gen 0 rate 10 depth 3 token 0 deadline 502220861525 time 502220971667 state 0x2 ref 0 00000100:00100000:3.0:1789762978.945211:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762978.945256:0:7595:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015191680 00000100:00100000:1.0:1789762978.945266:0:7595:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015191680 00000100:00100000:1.0:1789762978.945316:0:7595:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:1.0:1789762978.945329:0:7595:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e8342f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+6:6050:x1876702015191680:12345-192.168.203.48@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789762978.945344:0:7595:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789762978.945380:0:7595:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c462886a/0x416193e012b06113 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd973 expref: 6 pid: 6647 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.945400:0:7595:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789762978.945457:0:7595:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e8342f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:f1bd28de-ee78-4d18-bad3-fa7cec968893+5:6050:x1876702015191680:12345-192.168.203.48@tcp:103:ldlm_bl.0 Request processed in 132us (253us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762978.945464:0:7595:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 41 00000100:00100000:2.0:1789762978.953370:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762978.953410:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015191808 00000100:00100000:3.0:1789762978.953416:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015191808 00000100:00100000:3.0:1789762978.953462:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 401 00000100:00100000:3.0:1789762978.953474:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e84a41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+9:12244:x1876702015191808:12345-192.168.203.48@tcp:101:bash.0 00010000:00010000:3.0:1789762978.953490:0:5804:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:0.0:1789762978.953497:0:5801:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789762978.953507:0:5804:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b063e4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd97a expref: -99 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.953564:0:5804:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1000/0x416193e012b063eb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.953575:0:5804:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1000/0x416193e012b063eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.953583:0:5804:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1000/0x416193e012b063eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762978.953614:0:5804:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x41:0x0]: returning size 16384 00000004:00200000:3.0:1789762978.953617:0:5804:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789762978.953629:0:5804:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1000/0x416193e012b063eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5804 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.953647:0:5804:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b063e4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd97a expref: 11 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.953658:0:5804:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1000/0x416193e012b063eb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x41:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd97a expref: 10 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.953663:0:5804:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:3.0:1789762978.953666:0:5804:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789762978.953668:0:5804:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e87ca1000, rc 0) 00000100:00100000:3.0:1789762978.953700:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e84a41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:12244:x1876702015191808:12345-192.168.203.48@tcp:101:bash.0 Request processed in 231us (332us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762978.953708:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 401 00000100:00100000:0.0:1789762978.957137:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762978.957179:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015192064 00000100:00100000:1.0:1789762978.957184:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015192064 00000100:00100000:1.0:1789762978.957212:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 402 00000100:00100000:1.0:1789762978.957218:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f773b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+10:15903:x1876702015192064:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:1.0:1789762978.957227:0:5805:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789762978.957236:0:5805:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424a00/0x416193e012b063f2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd981 expref: -99 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789762978.957290:0:5805:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x41:0x0]: returning size 16384 00010000:00010000:1.0:1789762978.957470:0:5805:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424a00/0x416193e012b063f2 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd981 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.957480:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:1.0:1789762978.957481:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789762978.957484:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424a00/0x416193e012b063f2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd981 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.957493:0:5805:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e83424a00, rc 0) 00000100:00100000:1.0:1789762978.957526:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f773b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15903:x1876702015192064:12345-192.168.203.48@tcp:101:rm.0 Request processed in 308us (390us total) trans 4294967647 rc 301/301 00000100:00100000:1.0:1789762978.957532:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 402 00000100:00100000:0.0:1789762978.959076:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762978.959135:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015192192 00000100:00100000:1.0:1789762978.959140:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015192192 00000100:00100000:1.0:1789762978.959168:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 130 00000100:00100000:1.0:1789762978.959174:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981ea7ad7050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15903:x1876702015192192:12345-192.168.203.48@tcp:37:rm.0 00000100:00100000:1.0:1789762978.960972:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981ea7ad7050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15903:x1876702015192192:12345-192.168.203.48@tcp:37:rm.0 Request processed in 1797us (1895us total) trans 0 rc 0/0 00000100:00100000:1.0:1789762978.960978:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 130 00000100:00100000:2.0:1789762978.962347:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762978.962380:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015192448 00000100:00100000:3.0:1789762978.962385:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015192448 00000100:00100000:3.0:1789762978.962413:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 131 00000100:00100000:3.0:1789762978.962418:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e84a41880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+11:15903:x1876702015192448:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762978.962535:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e84a41880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15903:x1876702015192448:12345-192.168.203.48@tcp:35:rm.0 Request processed in 114us (186us total) trans 4294967648 rc 0/0 00000100:00100000:3.0:1789762978.962544:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 131 00000100:00100000:0.0:1789762978.963428:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:2.0:1789762978.963469:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015192704 00000100:00100000:2.0:1789762978.963476:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015192704 00000100:00100000:2.0:1789762978.963515:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 403 00000100:00100000:2.0:1789762978.963524:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f772d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+12:15903:x1876702015192704:12345-192.168.203.48@tcp:101:rm.0 00010000:00010000:2.0:1789762978.963534:0:6611:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789762978.963543:0:6611:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86dce200/0x416193e012b06400 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xd9e43f979a7bd988 expref: -99 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.963581:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86dcf800/0x416193e012b06407 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.963590:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e86dcf800/0x416193e012b06407 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x41:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.963603:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e86dcf800/0x416193e012b06407 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x41:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789762978.963610:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x41:0x0]: returning size 16384 00010000:00010000:2.0:1789762978.963657:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86dce200/0x416193e012b06400 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd988 expref: 15 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.963674:0:6611:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff981e86dcf800/0x416193e012b06407 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x41:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd988 expref: 14 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.963683:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:2.0:1789762978.963686:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1789762978.963688:0:6611:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff981e86dcf800, rc 0) 00000100:00100000:2.0:1789762978.963725:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f772d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15903:x1876702015192704:12345-192.168.203.48@tcp:101:rm.0 Request processed in 202us (297us total) trans 4294967649 rc 0/0 00000100:00100000:2.0:1789762978.963731:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 403 00000100:00100000:2.0:1789762978.964619:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762978.964657:0:5804:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015192832 00000100:00100000:3.0:1789762978.964663:0:5804:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015192832 00000100:00100000:3.0:1789762978.964688:0:5804:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 404 00000100:00100000:3.0:1789762978.964694:0:5804:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e84a40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15903:x1876702015192832:12345-192.168.203.48@tcp:41:rm.0 00000004:00000024:3.0:1789762978.964723:0:5804:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 6815744 pending 0 free 7614758912 avail 7603224576 00000100:00100000:3.0:1789762978.964755:0:5804:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e84a40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15903:x1876702015192832:12345-192.168.203.48@tcp:41:rm.0 Request processed in 61us (137us total) trans 0 rc 0/0 00000100:00100000:3.0:1789762978.964761:0:5804:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 404 00000100:00100000:0.0:1789762978.965553:0:3265:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762978.965593:0:5808:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015193088 00000100:00100000:1.0:1789762978.965598:0:5808:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015193088 00000100:00100000:1.0:1789762978.965628:0:5808:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 132 00000100:00100000:1.0:1789762978.965634:0:5808:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f773800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15903:x1876702015193088:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:1.0:1789762978.965778:0:5808:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f773800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15903:x1876702015193088:12345-192.168.203.48@tcp:35:rm.0 Request processed in 143us (226us total) trans 4294967650 rc 0/0 00000100:00100000:1.0:1789762978.965783:0:5808:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 132 00000100:00100000:0.0:1789762978.966518:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:1.0:1789762978.966553:0:5805:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015193344 00000100:00100000:1.0:1789762978.966558:0:5805:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015193344 00000100:00100000:1.0:1789762978.966581:0:5805:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 405 00000100:00100000:1.0:1789762978.966586:0:5805:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f772a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+15:15903:x1876702015193344:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:1.0:1789762978.966606:0:5805:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:1.0:1789762978.966618:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880ab800/0x416193e012b06105 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x43:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd96c expref: 15 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.966630:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e880aba00/0x416193e012b05e26 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x43:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd965 expref: 14 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.966637:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x43:0x0].0x0 (ffff981e9272cc00) 00010000:00010000:1.0:1789762978.966647:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1000/0x416193e012b063eb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x41:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd97a expref: 13 pid: 5804 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.966652:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:1.0:1789762978.966655:0:5805:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789762978.966663:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425200/0x416193e012b06415 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.966670:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425200/0x416193e012b06415 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.966675:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425200/0x416193e012b06415 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.966683:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425800/0x416193e012b0641c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.966688:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425800/0x416193e012b0641c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.966693:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425800/0x416193e012b0641c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.966910:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424200/0x416193e012b06423 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x43:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.966917:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424200/0x416193e012b06423 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x43:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.966922:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424200/0x416193e012b06423 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x43:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789762978.967705:0:5805:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].27, 64 off9920 00000004:00080000:1.0:1789762978.967713:0:5805:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 502271625203ns, next at 503271623910ns, rc = 0 00010000:00010000:1.0:1789762978.967746:0:5805:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425c00/0x416193e012b0642a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x43:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967756:0:5805:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425c00/0x416193e012b0642a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967762:0:5805:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425c00/0x416193e012b0642a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967767:0:5805:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425c00/0x416193e012b0642a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967772:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425c00/0x416193e012b0642a lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967776:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425c00/0x416193e012b0642a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967791:0:5805:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789762978.967800:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424200/0x416193e012b06423 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x43:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967804:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424200/0x416193e012b06423 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x43:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967809:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424200/0x416193e012b06423 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x43:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.967810:0:5794:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789762978.967818:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83424200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:2.0:1789762978.967819:0:5794:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425c00/0x416193e012b0642a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967822:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424200/0x416193e012b06423 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x43:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.967835:0:5794:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83425c00 already unused, calling callback (ffffffffc176c350) 00010000:00010000:1.0:1789762978.967837:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424200/0x416193e012b06423 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x43:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.967840:0:5794:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425c00/0x416193e012b0642a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x43:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967863:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x43:0x0].0x0 (ffff981ec2179400) 00010000:00010000:2.0:1789762978.967864:0:5794:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425c00/0x416193e012b0642a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x43:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967869:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789762978.967874:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424200/0x416193e012b06423 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x43:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967880:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e83424200/0x416193e012b06423 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x43:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.967881:0:5794:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x43:0x0].0x0 (ffff981ec2179400) 00010000:00010000:2.0:1789762978.967890:0:5794:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789762978.967891:0:5805:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425200/0x416193e012b06415 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.967894:0:5794:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425c00/0x416193e012b0642a lrc: 1/0,0 mode: --/PW res: [0x200000401:0x43:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967899:0:5805:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425200/0x416193e012b06415 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.967905:0:5794:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425c00/0x416193e012b0642a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x43:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967908:0:5805:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425200/0x416193e012b06415 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967916:0:5805:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83425200 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762978.967919:0:5805:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425200/0x416193e012b06415 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967924:0:5805:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425200/0x416193e012b06415 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789762978.967924:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762978.967927:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789762978.967931:0:5794:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789762978.967935:0:5805:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:1.0:1789762978.967938:0:5805:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762978.967940:0:5805:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425200/0x416193e012b06415 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.967943:0:5805:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425200/0x416193e012b06415 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x41:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789762978.967953:0:5805:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001e1279a8 reply state 00000000592addfb transno 4294967651 ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425800/0x416193e012b0641c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789762978.967958:0:5805:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789762978.967990:0:5805:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f772a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15903:x1876702015193344:12345-192.168.203.48@tcp:36:rm.0 Request processed in 1403us (1473us total) trans 4294967651 rc 0/0 00000100:00100000:1.0:1789762978.967995:0:5805:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 405 00010000:00010000:1.0:1789762978.968495:0:3257:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425800/0x416193e012b0641c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.968502:0:3257:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425800/0x416193e012b0641c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.968506:0:3257:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425800/0x416193e012b0641c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.968510:0:3257:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e83425800 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762978.968513:0:3257:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425800/0x416193e012b0641c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.968520:0:3257:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425800/0x416193e012b0641c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.968529:0:3257:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x4009f790 (ffff981ec2179e00) 00010000:00010000:1.0:1789762978.968532:0:3257:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762978.968536:0:3257:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425800/0x416193e012b0641c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.968542:0:3257:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e83425800/0x416193e012b0641c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x41:0x0].0x4009f790 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5805 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789762978.968961:0:3266:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762978.968995:0:11718:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015193600 00000100:00100000:3.0:1789762978.969055:0:11718:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015193600 00000100:00100000:3.0:1789762978.969115:0:11718:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 133 00000100:00100000:3.0:1789762978.969126:0:11718:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e9f772680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:15903:x1876702015193600:12345-192.168.203.48@tcp:35:rm.0 00000100:00100000:3.0:1789762978.969213:0:11718:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e9f772680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15903:x1876702015193600:12345-192.168.203.48@tcp:35:rm.0 Request processed in 90us (253us total) trans 4294967652 rc 0/0 00000100:00100000:3.0:1789762978.969220:0:11718:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 133 00000100:00100000:2.0:1789762978.969906:0:3264:0:(events.c:340:request_in_callback()) peer: 12345-192.168.203.48@tcp (source: 12345-192.168.203.48@tcp) 00000100:00100000:3.0:1789762978.969944:0:6611:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876702015193856 00000100:00100000:3.0:1789762978.969949:0:6611:0:(service.c:2366:ptlrpc_server_handle_req_in()) got req x1876702015193856 00000100:00100000:3.0:1789762978.969976:0:6611:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.48@tcp, seq: 406 00000100:00100000:3.0:1789762978.969983:0:6611:0:(service.c:2552:ptlrpc_server_handle_request()) Handling RPC req@ffff981e84a43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+14:15903:x1876702015193856:12345-192.168.203.48@tcp:36:rm.0 00010000:00010000:3.0:1789762978.970003:0:6611:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1789762978.970013:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e86dcf800/0x416193e012b06407 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x41:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd988 expref: 14 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970026:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec05c00/0x416193e012b05e0a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x41:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd95e expref: 13 pid: 6611 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970050:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e9272ca00) 00010000:00010000:3.0:1789762978.970064:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e8ec04800/0x416193e012b05da8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.48@tcp remote: 0xd9e43f979a7bd949 expref: 12 pid: 5805 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970075:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c300) 00010000:00010000:3.0:1789762978.970082:0:6611:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789762978.970121:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b06431 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970131:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b06431 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970141:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b06431 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970155:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0600/0x416193e012b06438 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970160:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0600/0x416193e012b06438 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970169:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0600/0x416193e012b06438 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970349:0:6611:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1e00/0x416193e012b0643f lrc: 3/0,1 mode: --/EX res: [0x200000401:0x41:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970355:0:6611:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1e00/0x416193e012b0643f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x41:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970361:0:6611:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1e00/0x416193e012b0643f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x41:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789762978.970731:0:6611:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x41:0x0]: returning size 16384 00010000:00010000:3.0:1789762978.970741:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1e00/0x416193e012b0643f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x41:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970749:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1e00/0x416193e012b0643f lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x41:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970755:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1e00/0x416193e012b0643f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x41:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970761:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87ca1e00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762978.970765:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1e00/0x416193e012b0643f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x41:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970772:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1e00/0x416193e012b0643f lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x41:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970781:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x41:0x0].0x0 (ffff981e9272ce00) 00010000:00010000:3.0:1789762978.970784:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789762978.970789:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1e00/0x416193e012b0643f lrc: 2/0,0 mode: --/EX res: [0x200000401:0x41:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970795:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca1e00/0x416193e012b0643f lrc: 0/0,0 mode: --/EX res: [0x200000401:0x41:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970812:0:6611:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b06431 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970819:0:6611:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b06431 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970826:0:6611:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b06431 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970832:0:6611:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87ca0a00 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:3.0:1789762978.970835:0:6611:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b06431 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970841:0:6611:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b06431 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970847:0:6611:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff981e9272c200) 00010000:00010000:3.0:1789762978.970849:0:6611:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789762978.970851:0:6611:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b06431 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789762978.970855:0:6611:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0a00/0x416193e012b06431 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789762978.970863:0:6611:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f8803a6c reply state 00000000eb3e5259 transno 4294967653 ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0600/0x416193e012b06438 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789762978.970872:0:6611:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789762978.970907:0:6611:0:(service.c:2610:ptlrpc_server_handle_request()) Handled RPC req@ffff981e84a43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:f1bd28de-ee78-4d18-bad3-fa7cec968893+13:15903:x1876702015193856:12345-192.168.203.48@tcp:36:rm.0 Request processed in 925us (1002us total) trans 4294967653 rc 0/0 00000100:00100000:3.0:1789762978.970912:0:6611:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.48@tcp, seq: 406 00010000:00010000:1.0:1789762978.971412:0:3258:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0600/0x416193e012b06438 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.971424:0:3258:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0600/0x416193e012b06438 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.971432:0:3258:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0600/0x416193e012b06438 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.971439:0:3258:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff981e87ca0600 already unused, calling callback (ffffffffc216e4f0) 00010000:00010000:1.0:1789762978.971442:0:3258:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0600/0x416193e012b06438 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.971449:0:3258:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0600/0x416193e012b06438 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.971457:0:3258:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x71fbd821 (ffff981e9272c800) 00010000:00010000:1.0:1789762978.971459:0:3258:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789762978.971461:0:3258:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0600/0x416193e012b06438 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762978.971466:0:3258:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff981e87ca0600/0x416193e012b06438 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x71fbd821 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6611 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789762979.768222:0:5793:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1789762979.958686:0:17667:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 503262594792ns, next at 503271623910ns, rc = 0 Debug log: 59260 lines, 59260 kept, 0 dropped, 0 bad.